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Developing and assessing academic and 
professional writing skills — An introduction 


Writing skills constitute one of the qualifications that are central both for 
academic and professional success. In an increasing number of occupa- 
tional fields, degree programmes and disciplines, writing skills are not only 
required in one’s mother tongue but also in English as the lingua franca 
of international communication. The observation that writing skills often 
do not develop to a sufficient extent if they are not fostered actively but 
rather considered a by-product of vocational or academic training has led 
to a number of initiatives such as, for example, the introduction of more 
process-oriented writing curricula, the establishment of writing centres as 
central support services at universities and the introduction of writing- 
intensive seminars in the disciplines. In higher education in Germany, a 
large number of writing centres and similar initiatives have been set up 
since 2011, when funding could be obtained for such measures from the 
German Federal Ministry of Education and Research (BMBF) in its “Qual- 
ity Pact for Teaching” programme. This programme addresses the needs 
of increasingly higher proportions of age cohorts that enrol in university 
degree programmes (in the German state of Hesse more than 50% in 2012) 
and that, due to diverse educational backgrounds, come with rather het- 
erogeneous entrance qualifications including writing skills. 

Writing curricula which are tailored to students’ specific needs cannot 
be developed without insight into the specific writing skills, for example, 
in terms of genre knowledge and languages, that these students will need 
in their education and training as well as their later professional lives. Fur- 
thermore, reliable assessment procedures are required which can be applied 
to determine what competencies students bring with them and can be con- 
sidered the foundation for further skills development, as well as to evaluate 
whether writing curricula yield the expected results. Given the importance 
of writing skills both for learning and knowledge construction as well as 
for academic and professional success, applied linguistics is faced with the 
challenge of informing the development of writing courses and writing- 
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intensive seminars in the disciplines and contributing to the establishment 
of best practises of literacy development and assessment. 

The present volume addresses the questions of how both academic and 
professional writing skills can be fostered under the conditions specified 
above and how writing skills development can be measured. These two cen- 
tral questions will be answered from three different perspectives reflected 
in the three parts of this volume: Part I “Vocabulary and terminology in 
academic writing”, which focuses on the lexical level; Part II “Complex 
writing competence constructs”, where a holistic perspective is taken trying 
to capture writing competence in all its complexity; and Part III “Subjec- 
tive conceptions of writing and how to foster it”, which considers students’ 
subjective attitudes and beliefs on writing, which, as will be shown, have 
an impact on the acceptance of writing support. The methods employed in 
the studies reported on range from corpus-linguistic approaches via analyti- 
cal and holistic assessment procedures to ethnographic studies. The writing 
assignments subjected to analysis range from seminar papers via personal 
portraits, letters of complaint and application to portfolios and statements 
on learning platforms. In the following, an overview of the structure of this 
volume and short summaries of each of the six contributions from four Euro- 
pean countries, Austria, Germany, Sweden and Switzerland, will be provided. 


Part I: Vocabulary and terminology in academic writing 


The two contributions of Part I address corpus-based studies that focus 
on lexical and terminological aspects of L2 academic writing in the fields 
of a) business administration and b) the natural sciences and engineering. 

In her article “Writing for specific purposes: Developing business stu- 
dents’ ability to ‘technicalize’”, Christine S. Sing (Vienna/Austria) inves- 
tigates how advanced students of business administration elaborate on 
technical terms in their L2 English writing by means of defining, exem- 
plifying and explaining, i.e., how they ‘technicalize’. For this purpose, she 
draws on a self-compiled specialized corpus, the corpus of Academic Busi- 
ness English (ABE), which comprises approximately one million tokens 
and was compiled from assessed university student writing on a broad 
thematic range of business topics. Christine Sing’s analyses focus on the 
challenge frequently encountered by ESP students that they may be familiar 
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with domain-specific concepts but lack the language resources necessary to 
appropriately embed them in their writing. Against this background, Sing 
pleads for a more effective orchestration of the teaching of subject-domain 
knowledge and the domain-specific language needed for communicating 
about it. This includes the necessity to firmly anchor professional genres 
in the relevant curricula, which should replace writing assignments, such 
as seminar papers, as they are often used as training texts. As she points 
out, the latter types of assignments “alienate students from the professional 
target genres” they are expected to be able to write in their later professional 
lives. Furthermore, she pleads for foregrounding the processual character 
of writing instead of misconstruing writing as a product. 

Hans Malmström, Diane Pecorari & Magnus Gustafsson (Gothen- 
burg & Växjö/Sweden) contribute to our understanding of what we may 
reasonably expect of English Medium Instruction (EMI) for the develop- 
ment of students’ English language proficiency. More specifically, their 
article “Coverage and development of academic vocabulary in assessment 
texts in English Medium Instruction” focuses on advanced students’ pro- 
ductive knowledge of English academic vocabulary at a technical univer- 
sity in Sweden where all degree programmes at graduate level use English 
as the medium of instruction. Drawing on a corpus comprising 80 texts 
(approximately 720,000 words) produced by Swedish and international 
Master of Science students in their first and second years of study, they 
set out to answer the following three research questions: 1. What is the 
lexical coverage of the students’ writing, i.e., what proportion of words in 
their texts is academic? 2. Are home students and international students 
comparable in terms of their productive academic vocabulary knowledge? 
And 3. Does students’ productive knowledge of academic words appear to 
develop during their studies? Their findings call the effectiveness of EMI 
for academic vocabulary development into question. 


Part II: Complex writing competence constructs 


Whereas the articles of Part I focus on one specific aspect of academic 
writing skills, lexical competence, Part II comprises two articles which ad- 
dress the assessment of professional and academic writing skills in a more 
encompassing manner. 
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In their article “Assessing writing in vocational education and train- 
ing schools: Results from an intervention study”, Liana Konstantinidou, 
Joachim Hoefele & Otto Kruse (Winterthur/Switzerland) describe an as- 
sessment procedure they used to evaluate a process-oriented approach to 
writing instruction in vocational education and training (VET) schools in- 
troduced in the German-speaking part of Switzerland. This new approach 
to writing instruction includes both German as L1 and German as L2 
approaches and was designed to more adequately cater to the diverse lan- 
guage needs in the multilingual, multi-ethnic vocational classroom than 
traditional product-oriented approaches. The article outlines the nature 
of the new approach to writing instruction and introduces the instruments 
used for the assessment of the students’ writing competence in a pre-test/ 
post-test/control group design with follow-up measurement including the 
writing tests and scoring procedures. Their main results show a significant 
increase in writing competence in the experimental group as compared to 
the control group. The article concludes with a discussion of the value of 
their assessment procedure. 

The construct of writing competence assessed in the pre-test/post-test 
study conducted by Susanne Göpferich & Imke Neumann (Giessen/Ger- 
many) and presented in their article “Writing competence profiles as an 
assessment grid? - Students’ L1 and L2 writing competences upon entering 
university and after one semester of instruction” is university students’ abil- 
ity to express themselves in a formally and linguistically correct, cohesive, 
coherent and well-reasoned manner in a genre they are familiar with from 
their secondary education. The empirical basis of their analyses is formed 
by the argumentative essays that 61 students (26 in their L1 German and 
35 in their L2 English) composed at the beginning and at the end of one- 
semester study skills academic writing courses in either their L1 German 
or their L2 English. These essays were subjected to both a (text-)linguis- 
tic error analysis and a holistic evaluation of their argumentative rigour 
yielding assessments in five areas: formal correctness, lexical correctness, 
syntactical correctness, text-linguistic correctness and argumentative rigour. 
The findings from both the beginning of the semester and the end of the 
semester are visualized in area charts, so-called writing competence profiles 
(WCP), which, against the background of dynamic systems theory, set in- 


dividual writing sub-competencies in relation to each other. These WCPs 
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show characteristic patterns, which are assumed to provide insight into how 
the individual sub-competencies and their developments may be related to 
one another. The characteristic patterns suggest that WCPs, when available 
for a larger range of writing competence levels, may be usable as grids for 
the assessment of writing competence. 


Part III: Subjective conceptions of writing and how to foster it 


The two articles combined in Part III address the subjective conceptions 
that university students have of ‘ideal’ professional and academic writing 
processes and their beliefs of how their writing skills can be fostered. 

In her contribution “Portfolios as a means of developing and assessing 
writing skills”, Sandra Ballweg (Bielefeld/Germany) explores a teacher’s 
and her engineering students’ actual use of a portfolio in the context of 
an academic writing course in German as a Foreign Language as well as 
their perception of this portfolio work. The empirical basis of the study is 
formed by 25 hours of audiotaped and transcribed lessons and portfolio 
conferences as well as interviews, four conducted with the teacher and three 
with each of seven participating students. Her objective is to reveal patterns 
of portfolio use and to generate hypotheses on the usefulness of portfolio 
work. The major hypothesis generated in this study through qualitative 
research methods relates to a gain-loss effect of portfolio use. The findings 
suggest that the introduction of portfolios in the writing classroom cannot 
just be viewed as an additional offer to the students and their learning but 
also necessitates abandoning established elements and procedures. There- 
fore, teachers have to make informed decisions as to both the focus of 
their teaching and the purpose of portfolio work. To enable them to do so, 
Ballweg pleads for preparing teachers for portfolio work and to support 
them in the process of employing it. 

Acknowledging that students’ beliefs about writing and writing strate- 
gies may interfere with their acceptance of writing support and that it is 
therefore helpful for writing instructors to be aware of these beliefs, Sabine 
Dengscherz & Melanie Steindl (Vienna/Austria) explore these beliefs us- 
ing statements posted by students on a learning platform. In their article 
“Prepare an outline first and then just write spontaneously’ — An analysis 
of students’ writing strategies and their attitudes towards professional writ- 
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ing”, they provide insights into students’ attitudes towards planning and 
spontaneous writing, both for short assignments in non-academic genres 
and longer texts in academic writing. For this purpose, statements from 
163 students were analysed, which were posted on the learning platform 
Moodle during four courses of the BA programme “Transcultural Com- 
munication” at the Centre for Translation Studies of the University of Vi- 
enna. The courses were designed around writing in German as an L1 or 
L2. The results reveal numerous individual writing strategies and beliefs 
about writing between a ‘conscious craft’ and a ‘kiss-of-the-muse’ position. 
Students who take the ‘conscious craft’ position are keen to learn more 
about writing strategies, whereas students with the ‘kiss of the muse’ posi- 
tion doubt whether writing support might be helpful for them at all. The 
article discusses the didactical potential of online forum discussions about 
writing and how they can be integrated into writing classrooms. 

We would like to thank all contributors for their excellent cooperation 
and hope that the present volume will give further impetus to the develop- 
ment of more student-centred approaches to professional and academic 
writing instruction and improved literacy assessment procedures. 

We also wish to thank the German Association for Applied Linguistics 
(GAL) for funding the publication of this volume, as well as our peer re- 
viewers for the expertise, time and effort that they invested into reviewing 
the articles of this volume and providing valuable suggestions for their 
optimization. We also wish to thank Lisa Beier for proof-reading all articles 
from the perspective of a native speaker, and the Peter Lang publishing 
team, especially Michael Ricker, for their excellent support in the pub- 
lishing process. Furthermore, support is acknowledged from the German 
Federal Ministry of Education and Research (BMBF), which has provided 
the funding for the establishment of the writing centre of Justus Liebig 
University, Giessen, in its “Quality Pact for Teaching” programme and thus 
also for staff that contributed to this volume. 


Giessen, January 2016 Susanne Gopferich & Imke Neumann 
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Christine S. Sing 
Vienna University of Economics and Business, Austria 


Writing for specific purposes: Developing 
business students’ ability to ‘technicalize’ 


Abstract 

English: This article examines business students’ ability to technicalize in an L2 
English writing task. Building up technicality in discourse is a key component of 
writing competence. Despite the importance of technicality for discipline-specific 
writing, so far little attention has been paid to identifying the usage patterns char- 
acteristic of this type of writing. The aim of this study is to investigate how under- 
graduate writers technicalize in elaborating on technical terms by means of defining, 
exemplifying and explaining. Drawing on a self-compiled specialized corpus, the 
study adopts a mixed-methods approach of computation and interpretation. It was 
found that technicalizing is a two-stage process, which consists of naming a term 
and subsequently embedding it in taxonomic relationships. The resulting chains of 
reference are taken to be indicative of field-specific uses in writing. The findings have 
important implications for developing business students’ writing skills in view of the 
conceptual challenges they meet in current specific-purpose instruction. 


German: Gegenstand des vorliegenden Beitrags ist die Art und Weise, wie Stud- 
ierende beim fachspezifischen Schreiben in ihrer L2 Englisch Technizitat (techni- 
cality) herstellen. Hierzu werden diejenigen sprachlichen Verfahren ermittelt, die 
sie nutzen, um Fachtermini durch Definitionen, Beispiele und Erklärungen in den 
Text einzubetten. Die Fähigkeit, Technizität herzustellen, wird dabei als wesentli- 
che Komponente der Kompetenz zum fachsprachlichen bzw. disziplinspezifischen 
Schreiben verstanden. Als Datengrundlage dient ein spezialisiertes Korpus von stu- 
dentischen Texten aus vier Bereichen der internationalen Betriebswirtschaftslehre, 
die auf Englisch als L2 verfasst wurden. In einem Mixed-Methods-Ansatz werden 
korpuslinguistische mit interpretatorischen Verfahren kombiniert. Es zeigt sich, dass 
das Herstellen von Technizität als zweistufiger Prozess beschrieben werden kann, in 
dem ein Fachausdruck zunächst benannt und dann in eine taxonomische Beziehung 
eingebettet wird, wodurch Referenzketten entstehen, die für die untersuchten Texte 
charakteristisch sind. Aus den gewonnenen Erkenntnissen werden Schlussfolgerun- 
gen für die Didaktik der Schreibkompetenzförderung in wirtschaftswissenschaftli- 
chen Lehr-/Lernkontexten gezogen. 
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1 Writing for specific purposes 


English for Specific Purposes (ESP) is traditionally conceived of as a “mate- 
rials- and teaching-led movement” (Dudley-Evans/St John 1998: 19), cater- 
ing to specific student needs in specific contexts. ESP is thus “an approach to 
language teaching in which all decisions as to content and method are based 
on the learner’s reason for learning” (Hutchinson/Waters 1987: 19). Recent 
developments in the field, however, have heightened the need for a fresh 
perspective on ESP: “What once looked to many like a straightforwardly 
needs-oriented, a- or pan-theoretical [...] approach, now, like the constantly 
changing learning targets it addresses, is itself becoming harder and harder 
to capture in anything like a single stop-action frame” (Belcher 2006: 134). 

The internationalization of higher education (HE) marks a sea change in 
the role of English in scholarship and instruction. English-medium instruc- 
tion has become the rule rather than the exception in tertiary education 
across Europe. Given the centrality of writing in HE institutions (Hyland 
2013), there is an increasing concern for students’ needs, lacks and wants 
in writing instruction. Indeed, it appears to be the case that students are 
lacking essential competencies when it comes to university writing and thus 
need to be acquainted with its primary purposes. This raises the question 
what differentiates university writing in English for General Academic Pur- 
poses (EGAP) from that in English for Specific Academic Purposes (ESAP) 
settings. 

Arguably, the transition from secondary to tertiary education represents 
a “cultural shift” (Gee 1996: 155) for most students as they meet with dif- 
ferent literacy expectations in their new environment. What is more, the 
novice writers face linguistic challenges in that they use English as an L2 in 
their writing, which means that they are struggling with both general and 
academic English at the tertiary level. Particularly, students are lacking in 
knowledge of the relevant target genres and are grappling with the highly 
formalized and conventionalized written academic style. 

In ESP settings, the writing task has regularly been of central impor- 
tance, most notably in business, academic and professional domains (Tardy 
2012: 6266). And yet, in specific-purpose writing instruction, these two 
challenges - cultural and linguistic - are compounded by what Peters et 
al. (2014: 744 ff.) refer to as “conceptual challenges”. The authors report 
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several pedagogical issues in ESP contexts, arising from the fact that learn- 
ing disciplinary knowledge and learning the specialist language are not 
carefully orchestrated. As a result, ESP students are not only linguistically 
challenged by having to overcome difficulties in both general academic 
English and the specialist language of their discipline; they also face con- 
ceptual challenges in that ESP programmes “require students to develop 
a more abstract understanding of concepts within the discipline, in order 
to be able to apply their knowledge effectively” (Peters et al. 2014: 755). 
This implies that ESP students may be familiar with field-specific concepts 
while lacking the language resources necessary to construe disciplinary 
knowledge in their writing. 

It is this conceptual challenge that will be addressed in the present analy- 
sis. It will be argued that the students in the ESP setting of a business school 
fail to be empowered by the potentially effective cross-fertilization of subject 
knowledge and disciplinary language. Building on Tribble’s (2002) contex- 
tual-analysis framework and Flowerdew’s (2004) parameters for specialized 
corpora, it will be shown that, in order to be able to engage in disciplinary 
discourse, student writers need to develop the ability to “technicalize” (Rav- 
elli 2004: 104), i.e., to grow aware of the linguistic resources highlighting 
that a given word or concept is embedded in a body of knowledge. 

Drawing on a self-compiled specialized corpus, the corpus of Academic 
Business English (ABE), this account sets out to examine three major modes 
of constructing technicality in business student writing, namely defining, 
explaining and exemplifying. Prior to presenting the main findings, I will 
first outline the theoretical framework of Systemic Functional Linguistics 
(SFL), in which the notions ‘technicality’ and ‘technicalize’ originated. The 
in-depth contextual analysis will conclude with a discussion of how specific- 
purpose writing instruction could address these conceptual challenges. 


2 Theoretical background 


The notions ‘technicality’ and ‘technicalizing’ are deeply embedded in the 
SFL framework, which contends that language users represent experi- 
ence through language by means of various genres and resources. Martin 
(1991: 308 f.) proposes a cross-classification of genres (report, explana- 
tion, exposition) and discourse functions (describing, explaining). In this 
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approach, the complexity of written texts can be described on the basis 
of two functional dimensions, technicality and abstraction. The rationale 
behind this classification is that, while both sciences and humanities rely on 
these genres and functions, there are clear discipline-specific preferences as 
to which features predominate. A case in point is the genre of report which, 
in science discourses, fulfils a taxonomizing function, while reports in the 
humanities tend to generalize. Not only does this highlight the instrumental 
character of discourses, it also illustrates the process of knowledge construc- 
tion in different disciplines. Different fields name, order and classify similar 
phenomena differently. 

This begs the question as to the linguistic resources that are instrumental 
in construing disciplinary knowledge in writing. In order to achieve rep- 
resentation, language typically fulfils three functions: 1. creating technical 
vocabulary, 2. classifying the experiential world, and 3. explaining the 
experiential world. Ravelli (2004: 104) argues that, in order to perform 
these functions, “writers must be able to give names to things, and to con- 
nect these names to each other, in order to theorize about the world around 
them”. She refers to these functional modes as technicalizing and rational- 
izing, respectively. Here the emphasis will be placed on technicalizing or 
theorizing, as it may also be referred to. It will be argued that, once capable 
of technicalizing, student writers demonstrate a deeper understanding of 
taxonomies, i.e., of how terms or concepts are to be placed in an ordered 
system. 

Technicalizing, i.e., the process of building up technicality in writing, 
involves two stages: 1. naming the phenomenon and 2. making the name 
technical. The latter is aided by ‘discourse cues’ such as textual signals or 
macrostructures. Woodward-Kron (2008: 238 f.) identified several linguis- 
tic devices that may be used to flag technicality in discourse, examples being 
definitions and taxonomic relationships. The reliance on discourse cues, 
however, presupposes the existence of an engaging writer-reader relation- 
ship. Since the English language represents a “writer-responsible culture” 
(Dahl 2004), strategies such as “lexical familiarization” are particularly 
rewarding. The latter can be defined as “a contextual aid, intentionally and 
explicitly provided by the author when writing for a specific readership. 
The writer’s intention is to help his reader by providing him with sufficient 
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familiarity with the new word, as employed in its context, so that the reader 
can continue reading with understanding” (Bramki/Williams 1984: 170). 

Taking the writer and reader community as a point of departure, Hy- 
land (2010) conceives of academic writing as an interactive communicative 
practice. Of particularly strong appeal is Hyland’s (2005: 37) interpersonal 
model of metadiscourse, i.e., “the cover term for the self-reflective expres- 
sions used to negotiate interactional meanings in a text, assisting the writer 
(or speaker) to express a viewpoint and engage with readers as members of 
a particular community”. More specifically, he distinguishes between two 
dimensions of interaction, namely interactional and interactive metadis- 
course. While the former is primarily concerned with reader engagement, 
it is the category of interactive metadiscourse that includes the linguistic 
resources instrumental in technicalizing. According to Hyland (2005: 37), 
interactive metadiscourse is grounded in the writer’s awareness of an audi- 
ence. For this reason, writers rely on resources such as transitions, frame 
markers and code glosses. The latter, which “supply additional information, 
by rephrasing, explaining or elaborating what has been said, to ensure the 
reader is able to recover the writer’s intended meaning” (Hyland 2005: 52), 
are vital for building up technicality in discourse. Table 1 presents the code 
glosses that may be used to signal technicality in discourse. 


Table 1: Technicalizing functions of code glosses 


Discourse function Linguistic resources Metadiscourse category 
Defining called, defined as, known as | Code glosses 
Exemplifying such as, for example, e.g. 

Explaining that is, which means 


In what follows, each of these functions will be subject to an in-depth analy- 
sis, involving both a quantitative analysis of their frequencies of occurrence 
and a more qualitative study of their discourse patterning. As will be shown, 
the student writers technicalize in that they establish so-called “chains of 
reference”, “i.e. sequences of noun phrases all referring to the same thing. 
This constitutes an important aspect of textual cohesion, which makes a 
text more than just a series of sentences” (Biber et al. 1999: 234 f.). The 
following corpus example showcases how such textual macrostructures 
are established: 
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[1] Iwill name three of these companies, explain what kind of products they sell 
and the materials used. [ABE_Business] 


The student clearly feels the need to first refer to the phenomenon, fol- 
lowed by embedding it in a given technical framework. Before this will be 
discussed, the data and context of analysis will be described. 


3 Data and method 
3.1 Corpus and context description 


Recent developments in the field of corpus-based analyses of student writing 
have seen the compilation of several corpora of (under-)graduate writing in 
English, for example, the Michigan Corpus of Upper-level Student Papers 
(MICUSP) and the corpus of British Academic Written English (BAWE). 
While both provide a rich source of data, comprising several genres and dis- 
ciplines, they need to be carefully differentiated from learner corpora on the 
one hand and Language for Specific Purposes (LSP) corpora such as the ABE 
corpus on the other. The former are clearly indebted to Second Language 
Acquisition (SLA) research, where the emphasis is placed on interlanguage 
features such as a limited lexical repertoire, semantic misuse or overuse 
of connective devices (Paquot 2010). The latter, by contrast, are referred 
to as specialized corpora, which “are corpora designed for the purpose of 
creating a sample of specialized language either by collecting texts of similar 
content [...] or of similar text-type or genre” (Gavioli 2005: 7). These tend 
to be small and localized collections of data, which means that scholars, 
like the author of this present study, are frequently forced to assemble their 
own corpora (Tognini-Bonelli 2008: 35). The ABE corpus, which makes 
up the database of the present study, is such a self-compiled, specialized 
corpus. As shown in Table 2, with a size of approximately one million 
tokens, the ABE corpus greatly exceeds the usual 250,000 word mark of 
specialized corpora. In actual practice, specialized corpora typically range 
between around 60,000 words and 250,000 words. Handford (2010: 258) 
thus concludes that “[i]n terms of actual size, a specialized corpus can be 
defined as large [...] if it contains a million words”. 
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Table 2: ABE corpus data 


Subcorpora | Number of Type/token ratio Number of running words 
papers (STTR) (tokens) 

Business 103 41.22 236,917 

Economics 104 40.76 273,455 

Finance 104 39.60 253,203 

Marketing 102 40.64 251,156 

ABE Total 413 40.55 1,014,731 


Given that a corpus is “not simply a collection of texts” (Biber/Conrad/ 


Reppen 1998: 246), there should be a principled stand on corpus compila- 


tion. The compilation of the ABE corpus was guided by a clear set of design 


criteria. These are summarized in Table 3 below, drawing on Flowerdew’s 


(2004: 21) set of parameters for defining specialized corpora on the one 
hand and Tribble’s (2002: 133) contextual-analysis framework on the other. 


Table 3: Description of data and context 


DATA DESCRIPTION 
Specific purpose for To investigate business student writing in a localized 
compilation setting 
Size Large specialized corpus (ABE) of approx. 1 million 


running words 


Language used 


Written academic English, used as L2 


CONTEXTUAL DESCRIPTION 


Genre family 


Assessed university student writing 


Communicative purpose 


To demonstrate / be accredited for proficiency 
in academic English and knowledge of business 
concepts 


Text type 413 simple assignments, seminar papers 

Subject matter Broad thematic range of business topics 

Setting ESP setting of a business school; committed to a 
wide-angled approach to the teaching of business 
English 

Participants Advanced undergraduate students of international 


business administration 
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In what follows, each of these features will be described, detailing the specif- 
ics of the ABE corpus and the institutional context in which it originated. 
First and foremost, “[h]aving a clearly articulated question is an essential 
step in corpus construction since it will guide the design of the corpus” 
(Reppen 2010: 31). And yet, the compilation of specialized corpora tends 
to be prompted by more specific purposes. The most obvious reason for 
building “DIY corpora” (McEnery/Xiao/Tono 2006: 21) lies in the fact that 
none of the existing corpora include a sample of the genre investigated. An- 
other main driving force behind self-compiled corpora are methodological 
considerations. Even if corpora are available, not all data lend themselves 
to conducting contextualized analyses. Studying business student writing 
with a view to enhancing these students’ development of writing skills ne- 
cessitates the retrievability of contextual information, ranging from details 
about the institutional setting and the curriculum in operation to localized 
academic practices. Thus, methodologically, small self-compiled corpora 
have the edge over larger corpora whenever detailed, fully contextualized 
analyses are intended (Nelson 2010: 55). Yet another reason for compiling 
the ABE consisted in creating a specialized corpus that would contain the 
type of language characteristic of this particular ESP setting. 

As mentioned earlier, the ABE corpus can be considered a large special- 
ized corpus. The question of corpus size is closely intertwined with the issue 
of representativeness. Unlike general corpora, for which representative- 
ness may be achieved through size, this “aim [...] is rendered impossible 
through the need to target a disciplinary or thematic specialty” (Williams 
2002: 45). Instead, “the representativeness of specialized corpora is usually 
measured by reference to external selection criteria (i.e. by/for whom the 
text is produced, what is its subject matter), which are regarded as some- 
what subjective” (Flowerdew 2004: 18; highlighting in the original). In 
order to mitigate the effects of subjectivity, the collection of data needs to 
meet clear sampling criteria. Accordingly, texts had to meet the following 
external criteria for inclusion in the ABE: All texts had to be complete rather 
than samples. They had to be single-authored and of equal length (i.e., not 
exceeding the word limit of 2,400 words). They had to be organized in 
terms of the macrostructure laid down in the institute’s style sheet. Their 
thematic focus was required to be on either business, economics, finance 


or marketing, which corresponded to the subject matter dealt with in the 
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seminars. These external criteria clearly limit the range of texts eligible for 
inclusion, meaning that “[b]y restricting the scope of the corpus, energy 
can be directed towards assembling a detailed collection of texts that fully 
represent the kind of academic language one is likely to encounter” (Meyer 
2002: 36). 

However, the conditions of data collection are highly unpredictable, 
particularly when student work is involved. Alsop & Nesi (2009: 73) report 
similar sampling problems when compiling the BAWE corpus. As they did 
not quite know what to expect from the assignments — neither in terms of 
volume nor nature — they used a matrix of disciplinary groupings on four 
different levels of study. Following Alsop & Nesi, the ABE corpus texts were 
sampled on the basis of a matrix of four disciplinary groupings in order to 
plan the structure of the corpus, i.e., the sub-fields of business, economics, 
finance and marketing served to organize corpus holdings. Accordingly, 
texts were cross-classified making up matrix cells, which were based on 
these sub-fields on the one hand and on the external criteria mentioned 
above on the other. In order to achieve balance, texts were then added to 
the corpus until all cells were filled in equal numbers. 

Prior to moving on to the description of context, a passing reference to 
the language used in the corpus texts is in order. The specialized language 
represented in the ABE corpus is “[t]he academic business English required 
by students on courses in disciplines such as business, finance, account- 
ing and banking [which] has more in common with the study of other 
EAP [English for Academic Purposes] disciplines” (Dudley-Evans/St John 
1998: 53). The student writing included in the ABE is therefore strongly 
informed by the institutional context. The close interweaving of texts and 
the institutional setting in which they originated warrants the distinction 
between text type, genre and communicative purpose, all of which relate to 
the categorization of written texts. The distinction between genre and text 
type, in particular, is a moot point. Broadly, the distinction is grounded in 
whether external or internal criteria are used for grouping texts together. 
As originally laid down by Biber (1988), genre “is defined as a category as- 
signed on the basis of external criteria such as intended audience, purpose, 
and activity type, that is, it refers to a conventional, culturally recognized 
grouping of texts based on properties other than lexical or grammatical 


(co-)occurrence features” (Lee 2001: 38). The category text type, by con- 
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trast, characterizes texts in terms of their textual properties and linguistic 
patterning. However, as pointed out by Paltridge (1996), the distinction is 
anything but watertight and is best seen as complementary. More precisely, 
genres can be realized by different text types, that is, “more than one genre 
may share the same type. [...] Equally, a single genre [...] may be associated 
with more than one text type” (Paltridge 1996: 239). 

In modelling the ‘seminar paper’ (SP) as a specific type of written text, 
the present account subscribes to Biber’s (1988) corpus-linguistic classifica- 
tion. Accordingly, the SP is viewed as a specific, institutionalized form of 
assignment writing, which a) has a particular organization, b) deals with a 
specialized topic and c) realizes an identifiable set of social and communi- 
cative functions. The SP is deeply rooted in German-speaking HE (Kruse 
2006). It is an argued text similar to an exposition, frequently taking on 
the form of research articles (RA) in miniature (Ehlich 2003). In many 
disciplines, particularly in the social sciences and humanities, it continues 
to be the standard type of assessed writing. In the context of Austrian HE, 
the seminar paper has been studied extensively (Gruber 2006). A case in 
point is Gruber’s (2004: 50) identification of the SP’s prototypical structure: 
introduction, problem definition, descriptive and argumentative discussion 
of the problem(s), case description and conclusion. 

Its generic properties are further determined by a detailed set of instruc- 
tions, according to which the paper is to present an ‘academic argument’ 
rather than make recommendations for (case-)specific situations. The strict 
formal criteria laid down for the SP typically relate to principles of layout 
and text organization, citation practices and bibliographical conventions. 
The SP is typically a simple assignment, i.e., the number of texts equals the 
number of writing tasks; conversely, the number of texts equals the num- 
ber of authors. Thematically, SPs are subject to sub-field specific variation. 
The internal criteria used to determine the SP as an abstract text type were 
topic-based. In order to ensure that the teaching and learning context is 
adequately reflected in the corpus data, the corpus was subdivided into four 
thematic subcorpora, each relating to the topical focus of seminar teaching: 
business studies, economics, finance and marketing. Regarding the social 
and communicative functions of SPs, they can be argued to realize a par- 
ticular genre, namely assessed student writing. In fact, university student 


writing has been shown to constitute a genre family consisting of various 
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subgenres (Nesi/Gardner 2012). In combining several of these subgenres, 
the SP fulfils what is perhaps the primary function of assessed university 
writing: accreditation (Nesi/Gardner 2012: 23). 

The setting in which the texts were collected and the participants can 
be specified as follows: The institutional context at hand is a fully-fledged 
academic unit, which, despite its full-departmental status, does not offer 
bona fide degree programmes. It instead caters to business programmes, 
such as the four-year diploma programme of International Business Admin- 
istration (IBA). Owing to the content-based curriculum, the focus clearly 
is on specialist language (Business English - BE), leaving little scope for 
developing general English language skills. Similarly, there are few oppor- 
tunities of familiarizing students with academic English, which means that 
the students will not have received any formal writing instruction prior to 
producing the assignment. In the light of this curricular framework, it is per- 
haps not surprising that a wide-angled approach (Basturkmen 2010: 143) 
to the teaching of BE is adopted, implying that English is taught through a 
variety of topics in several business fields. The integration of subject-specific 
knowledge with language learning is thus hinged on a terminology-driven 
approach to BE. The participants are usually in their third or fourth year 
of the diploma programme when producing the SP. They are referred to 
as advanced students in the sense that they will have spent at least three 
years in HE, in other words, ‘advanced’ does not describe their proficiency 
in English. As there are no initial placement tests nor any official language 
testing throughout the programme, no assessed information as to their 
actual level of English can be provided. Neither are there any detailed 
metadata available on individual student authors. The vast majority are 
however L1 users of (Austrian) German. At the same time, there is a large 
intake of international students, accounting for approximately 10% of the 
entire student population. 


3.2 Method 


Discipline-specific writing has mainly been studied from either a corpus- 
linguistic, variationist perspective or using ethnographic research methods. 
Corpus-based approaches aim to retrieve lexico-grammatical patterns in 
order to identify field-specific uses of language. These variationist accounts 
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tend to focus on variables such as novice vs. expert practices or English 
L1 vs. English L2 uses in discipline-specific writing. There is a substantial 
body of research which has examined disciplinary variation in academic 
discourse, including citation practices (Samraj 2013; Charles 2006), stance 
(Chang/Schleppegrell 2011; Biber 2006) and shell noun uses (Flowerdew 
2006; Aktas/Cortes 2008). However, approaches of this kind carry with 
them various well-known limitations. First, it seems to be the case that, due 
to the contrastive approach taken, variationist studies conceive of discipli- 
nary forms of writing as uniform practices. The inherent normativity then 
leads to an analytical focus “on identifying academic conventions [...] and 
on (or with a view to) exploring how students might be taught to become 
proficient or ‘expert’” (Lillis/Scott 2007: 13). Second, expert writing is fre- 
quently conceived of as constant and stable, ignoring the fact that lexical 
complexity in the writing of experts is itself subject to considerable variation 
over time (Trinh 2011). Thus, once both a timescale and a time window are 
taken into account, expert practices no longer serve as the reliable yardstick 
they are made out to be. Yet another shortcoming of this approach consists 
in the decontextualized lexico-grammatical patterns identified; more often 
than not, the analysis does not include the level of discourse, i.e., language 
beyond the sentence level. 

Ethnographically-oriented studies seek to address the social conditions of 
the production and interpretation of academic texts. Accordingly, Gnutz- 
mann & Rabe (2014: 33) observe “a certain neglect of the wider context 
of academic writing and publishing, since many conventions of disciplinary 
cultures are unwritten and have to be reconstructed through other methods”. 
Using interpretative methods, the authors propose an interview-based study 
of the language demands and attitudes of German researchers using English 
as an L2. The ultimate aim of such an account consists in homing in on pre- 
cisely those contextual factors amiss in much corpus-based research. 

The present account proposes a ‘third way’ of analysing ESP writing, in- 
tending to reconcile these two approaches. It combines the empirical rigour 
of corpus-based studies with the interpretative methods of ethnographical 
research. The study at hand thus provides a fully contextualized analysis 
in drawing on the ABE as a localized collection of data. Specialized cor- 
pora permit the analyst to study patterns of language use in the contexts 


in which they originated (Handford 2010: 258 f.). More precisely, with 
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specialized corpora, “the analyst is probably also the compiler and does 
have familiarity with the wider socio-cultural context in which the text was 
created [...]. The compiler-cum-analyst can therefore act as [...] specialist 
informant to shed light on the corpus data” (Flowerdew 2005: 329). Not 
least the growing complexity of ESP settings has highlighted that corpus 
data are invariably situated. The importance of studying corpus texts in 
their contextual environment results from the tight interweaving of text 
and context: “To make sense of the way particular texts and particular 
grammatical patterns occur, and why they occur in a particular order, it is 
essential to consider what is distinctive about texts within their particular 
institutional context” (Veel 1997: 162). The author of the present study 
is such an insider. This field knowledge will be used when discussing the 
findings of the corpus study below. 

In what essentially constitutes a mixed-methods approach involving 
computation and interpretation, the corpus data were subject to a bottom- 
up, inductive analysis, followed by the study of discourse patterns, which 
was more qualitative in nature. Using Wordsmith Tools 6 (Scott 2012), a 
wordlist analysis was first conducted for all four subcorpora. This served 
to retrieve the most frequently used code glosses across the ABE, as exempli- 
fied in Table 1 above. These instances were then concordanced to identify 
their most frequent collocates. In a second stage, the discourse patterns in 
which these code glosses are embedded were examined using MAXODA. 
For this purpose, the chains of reference were coded as definitions, ex- 
emplifications and explanations respectively. It should be noted that the 
classification of metadiscourse devices is problematic in itself, since these 
describe a continuum of functional resources rather than mutually exclusive 
categories. Therefore, metadiscourse is notoriously difficult to quantify, 
which is why the emphasis was placed on exploring the usage patterns of 
these discourse phenomena in the data at hand, paying particular attention 
to intra-disciplinary variation. 


4 Results and discussion 


The analysis set out to examine the linguistic resources used by business stu- 

dents to build up technicality in their writing. Technicalizing was introduced 

as a two-stage process: 1. referring to/naming the phenomenon concerned 
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and 2. making it technical by means of elaboration. Due to the interactive 
nature of academic writing, the first step of the analysis involved retrieving 
all relevant code glosses, assuming that they highlight instances of techni- 
cal language use. The emphasis was placed on glosses that mark defining, 
exemplifying and explaining. As can be seen in Fig. 1, which shows the 
main findings for this first set of analyses, the three metadiscoursal devices 
are used fairly differently across the subcorpora. 


un 
© 
= 
S 
= 
=) 
© 
2 
= 
m 
[0] 
a 
= 
v 
= 
Ö 
=] 
© 
© 
= 
LL 


Business Economics Finance Marketing 


E Definition Exemplification MExplanation 


Fig. 1: Distribution of code glosses across the ABE subcorpora 


Generally, it is evident that the construction of technicality chiefly occurs 
by means of exemplification and explanation. Explicit glosses used to signal 
acts of defining are clearly less frequent than those relating to exemplification 
and explanation. Referring more specifically to individual subcorpora and 
zooming in on the more subtle differences between them, several interest- 
ing findings were produced. First and foremost, the in-corpus variation is 
remarkable, given that all four ABE subcorpora are made up of the same 
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text type. While definitions clearly play a subordinate role in building up 
technicality in these corpus texts, the distribution of exemplifications and 
explanations in the corpus data varies greatly. Focusing on frequency of 
occurrence alone, it could be argued that the results for the business and 
economics subcorpora are fairly similar. However, when taking into account 
the relations between the most pervasive code glosses exemplification and 
explanation, the distribution for business and finance on the one hand, and 
economics and marketing on the other is fairly similar. In the latter, the stu- 
dent writers use slightly more exemplifications than explanations while the 
situation is reversed for the business and finance subcorpora. This can be 
accounted for by the differences in composition structure: While both busi- 
ness and finance papers tend to be organized as case studies and thus follow 
the problem-solution pattern, papers in economics and marketing tend to 
be argued essays, both descriptive and argumentative in nature, implying 
that the student writers rely on exemplifications to illustrate the points they 
make rather than explaining case specifics in detail. Even though these facts 
and figures of corpus analysis highlight important differences between the 
ABE subcorpora, they represent only a small part of a much bigger picture. 
Therefore, the second part of the analysis involved identifying the usage pat- 
terns and textual macrostructures in which these code glosses are embedded. 


4.1 Technicalizing by means of defining 


Due to their knowledge-telling function, definitions expand on a given term 
that has been ‘named’ in the preceding context. When labelling such a dis- 
course act, the student authors whose writing is documented in the ABE tend 
to resort to the following linguistic resources: called, the term X, defined as, 
define and known as, which makes them the top five most frequently used 
glosses of this category across the ABE subcorpora. The analysis revealed 
several usage patterns which will now be discussed (the subcorpora from 
which the examples are taken are indicated in square brackets): 
[2] AUTHOR and AUTHOR (1995: 17) define CC [= Corporate Citizenship] 


as the engagement of companies, using corporate resources in order to solve 
[ABE_Business] 


[3] type of tourism called events tourism. AUTHOR defines events tourism as 
“the planning, development [ABE_Marketing] 
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As shown, the most obvious way of introducing a new term is by what 
Adel & Garretson (2006: 273) refer to as “attribution”, the procedures 
by which “writers [...] frequently attribute statements and acts to other 
researchers”. The student writers, too, make use of this standard procedure 
by means of paraphrasing or quoting verbatim from a source, which means 
that the verb to define is used as a reporting verb. In both instances, how- 
ever, the term is not defined on first mention; instead it has already been 
used in the preceding context, which testifies to the existence of “chains of 
reference”. Accordingly, the term Corporate Citizenship has already been 
introduced as it is defined in abbreviated form. Example [3] provides an 
introductory sentence in which reference is made to the term event tourism, 
which is subsequently defined in a more technical way. 

Chains of reference are also operative in yet another pervasive pattern 
identified in business student writing. As exemplified below, the reference 
to a given term is established by the section heading that serves as the an- 
tecedent and immediately precedes the defining act: 


[4] Joint Ventures [= section heading] A joint venture can be defined as a com- 
pany which is owned corporately by two or more parties [ABE_Business] 


[5] Definitions [= section heading] The Role of Information [= section heading] 
Information can be defined as “data with attributes of relevance and purpose 
[ABE_Economics] 


Technicality is thus gradually built up in first naming the phenomenon in 
question, followed by according it a technical status. As indeed highlighted 
by the corpus examples above, technicalizing clearly has a taxonomizing 
function in that the term is positioned in an ordered system. The students’ 
taxonomic organization of their writing topic creates such a system, whose 
structure becomes manifest in the tables of contents of their papers. The fact 
that the student writers seem to be preoccupied with taxonomies rather than 
developing ideas is further supported by the relatively low lexical density 
(see STTR scores in Table 2 above), suggesting that their writing has a low 
informational value and that topic development occurs on the basis of few 
new language items. In their study of technicality in geography textbooks, 
Wignell, Martin & Eggins (1993: 157 ff.) found that chapter or (sub-)sec- 
tion headings and the emboldening of terms correspond to “superordina- 
tion taxonomy” in contrast to “composition taxonomy”. The former is 
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primarily concerned with classifying a given term while the latter provides a 
list of its elements. For example, the superordination taxonomy of the tech- 
nical term climate would include different types of climate, e.g., tropical, 
moderate and dry, while the composition taxonomy would include a list of 
descriptive elements such as temperature, pressure systems or atmospheric 
moisture (Wignell/Martin/Eggins 1993: 159). 

It can be argued that the above examples are indicative of a strong con- 
cern for superordination rather than composition taxonomy. The findings 
also suggest that students are concerned with composition taxonomy as 
illustrated by the following examples: 


[6] cross-cultural cooperation in a positive manner. The first dimension is called 
“universalism versus particularism [ABE_Economics] 


[7] change process is called “managing the transition”. The third phase is called 
“sustaining momentum”. [ABE_Finance] 


Taxonomizing presupposes that students are capable of identifying tech- 
nical vocabulary in the sources used. In so doing, they make use of the 
composite nature of technical language and seek to decompose it by fore- 
grounding component parts such as step, dimension, phase, stage or mode. 
This has the effect of an unfolding discourse, which can be achieved in two 
directions: from naming to signalling or from signalling to naming. 

The following examples show a reversal of the process of technicality 
construction: 


[8] long a put option. This strategy is called protective put and limits the maxi- 
mum loss to the premium [ABE_Finance] 


[9] expatriates normally return back home, this process is called repatriation. 
Cost of foreign assignments [ABE_Business] 


Here the description of the phenomenon in question occurs first, followed 
by the introduction of the terms, namely protective put and repatriation. In 
this pattern, the chains of references are established by means of so-called 
‘signalling nouns’ or ‘shell nouns’ (Sing 2013). “A signalling noun is po- 
tentially any abstract noun which is unspecific out of context, but specific 
in context” (Flowerdew 2006: 348). Nouns such as attitude, difficulty and 
process are pervasive in academic discourse, performing important cohesive 
functions within or between sentences. 
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The preceding discussion has clearly shown that these student writ- 
ers, despite imperfections of language and style, demonstrate the ability 
to technicalize. However, they do not seem to conceive of themselves as 
‘knowledge-makers’. 

[10] In the third part of my paper I will try to define the term hooliganism and 


give some basic explanations for the phenomenon hooliganism. Finally, in 
the last part [ABE_Business] 


[11] This section will define the term new media and then shortly discuss the 
change in language culture. [ABE_Marketing] 


[12] The first part basically outlines the aim of the agreement and defines the term 
‘trade in services’. The second part gives an overview [ABE_Economics] 


In contrast to the uses of to define as a reporting verb above, these uses 
highlight that defining proper is restricted to attribution. Although the stu- 
dent writers use the first person pronoun I, they do not perform the act of 
defining. Instead the main emphasis is placed on the paper’s organization, as 
evidenced by the co-occurrence with so-called frame markers such as first, 
then, finally and in the last part, which “can be used to sequence parts of 
the text or to internally order an argument” (Hyland 2005: 51). This means 
that, in this particular pattern, the verb serves as a genuine metatextual 
gloss. Alternatively, the students make use of non-human agents (section, 
part), which also frame, rather than perform, the defining act. 

One possible explanation for this pattern is students’ awareness of the 
genre, i.e., assessed university writing, and its main social purpose, accredi- 
tation. The student writers thus show compliance and do what is expected 
of them. Given that the business students will not have received any writing 
instruction prior to producing the assignment, they come equipped with 
lay beliefs about what constitutes good academic writing. The fact that 
they are still in the process of internalizing their knowledge is shown in the 
following set of examples: 

[13] Before examining the topic in detail, it is necessary to define the reoccur- 


ring terms ‘endorsement marketing’ and ‘celebrity endorser’. AUTHOR 
(2009: 126) states that endorsement [ABE_Marketing] 


[14] First it is important to define the term that is probably the most important in 
the whole paper, ethnic minority. Since there is no general [ABE_Marketing] 
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These examples highlight that the patterning is clearly learning-related. It is 
evident that these business students are anxious to define key terminology 
in their writing. At the same time, they are overwhelmed by the task, given 
the abundance of definitions for relevant concepts. They find it difficult to 
tolerate this ambiguity, seeking to reduce the complexity to the one-and- 
only correct definition of a given phenomenon. 

[15] confused and are used in a much diversified sense (AUTHOR 2004: 3). For 


this reason, it is important to define e-commerce in comparison to e-business 
and illustrate the differences. [ABE_Marketing] 


[16] paper is dealing with cultural aspects of the COUNTRY capital, it seems to be 
a good approach to define more narrowly the use of the word culture. Culture 
is certainly one of the two or three most complicated words [ABE_Business] 


All in all, the business students at hand tend to be familiar with different 
modes of elaborating on technical terms, thereby establishing chains of 
reference to build up technicality. The students are also clearly alert to 
expectations in terms of genre conventions. With a view to complying 
with the requirements of the writing task, the student writers have inter- 
nalized that, in academic writing, they are to define key words prior to 
discussing them in more detail. In the sections that follow, the findings for 
the modes of technicalizing by means of exemplification and explanation 
will be presented. 


4.2 Technicalizing by means of exemplifying 


Technicality may also be constructed when writers elaborate on a term 
by means of exemplification. The most pervasive metadiscoursal devices 
across the ABE subcorpora are: such as, for example, include, e.g. and 
for instance. Crucially, there is considerable variation with regard to the 
concepts that are exemplified in discourse. As shown in Table 4, the above- 
mentioned metadiscoursal resources co-occur with both field-specific (e.g., 
companies, products, investors) and ‘signalling noun’ (e.g., factors, issues, 
problem) L1 collocates (listed in terms of their frequency, with a minimum 
occurrence of 4): 
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Table 4: L1 collocates across subcorpora (minimum frequency of 4 occurrences) 


Corpora L1 collocates 

ABE Business | issues, values, activities, factors, problem, countries 

ABE countries, factors, companies, issues, problems, products, services 
Economics 


ABE Finance | instruments, investors, areas, failures, products, sectors, banks, 
countries, factors, institutions, events 


ABE events, attributes, countries, technologies, activities, areas, 
Marketing communication, industries, media, problems, services 


As shown below, exemplifying proper serves to elaborate field-specific or 
procedural vocabulary. The following examples illustrate this point: 


[17] Patents require inventions. In general, the main characteristics of inventions 
such as novelty, inventiveness, [ABE_Economics] 


[18] Equity capital markets deal with stocks and derivative instruments, such 
as forward contracts, futures, and options (AUTHOR 2011: 14). [ABE_Fi- 
nance| 


If field-specific vocabulary is exemplified, the term will most likely have 
been introduced in the preceding context, which may also involve attribu- 
tion. What is more, exemplification may also be used to instantiate tax- 
onomies. Following on from the discussion in Section 4.1, the effect of 
superordination taxonomy on subsection headings is also noticeable here. 
[19] Recruitment [= section heading] Recruitment and selection is more than just 


hiring people. Pre-recruitment activities include writing a job description as 
well as developing [ABE_Business] 


[20] Introduction [= section heading] There are many forms of business commu- 
nication, such as marketing or image communication, crisis communication 


[ABE_Marketing] 


Interestingly, the effect of composition taxonomy on how students elaborate 
a given term by means of exemplification seems to be weaker than when 
they technicalize by means of defining resources. More importantly still, ex- 
emplification appears to encourage students to opt for chains of references 
in which ideas are loosely strung together, enforcing a linear, associative 
structuring rather than building taxonomies. 
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[21] There exist different types of tour operators- such as ‘niche tour operators’, who 
specialize in certain destinations as for example a region [ABE_Marketing] 


[22] (compound options) and options on swaps (swaptions). Compound options, 
for instance, appear in four basic forms: call on a call, call on a put, put on 
a call and put on a put. [ABE_Finance] 


The virtual absence of hierarchical structures in exemplifying chains of 

reference may also point to another task-related requirement. The SP is to 

show students’ ability to reason. Witness the following corpus examples: 
[23] sponsors, suppliers and employees. Secondary stakeholders are, for exam- 


ple, the host community and media. In addition, the external environment 
[ABE_Marketing] 


[24] The terms have also been used to invoke a variety of associated concepts. For 
instance, exploration has been linked to radical innovation, market expan- 
sion [ABE_Finance] 


Of course, the student writers cannot be assumed to construct a discursive 
argument in the examples above. Contrasting different types of stakehold- 
ers on the basis of exemplification is conceptually not very challenging. 
And yet, while still remaining descriptive on the surface, the students show 
initial signs of truly elaborating rather than simply naming a given term. 


4.3 Technicalizing by means of explaining 


The corpus analysis revealed that the top five most frequently used code 
glosses in this category are means that, i.e., explain, namely and refers to. 
By and large, the usage patterns retrieved are fairly consistent with those 
previously identified. In general, it seems that students perceive of the reper- 
toire of discourse cues available to them as alternative modes of construing 
technicality, while being largely unaware of the more subtle differences be- 
tween them. In what follows, I will first briefly comment on the established 
patterns and subsequently pinpoint the causes for variation between them. 

In explaining a given term, students also establish chains of reference in 
which the phenomenon in question is first named and only subsequently 
explained more accurately, frequently involving attribution (see examples 
[25] and [26]). 

[25] reduce their carbon footprint, i.e., “the total greenhouse gas emissions caused 

directly and indirectly [ABE_Business] 
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[26] The term mass tourism was born. Mass tourism is characterized by “short- 
term travel of non-residents [ABE_Economics] 


On the level of taxonomies, the effects of both superordination ([27] and 
[28]) and composition taxonomy ([29] and [30]) are comparable to those 
identified for elaboration. 

[27] Methods and techniques of traffickers [= section heading] Ways of recruit- 


ment [= subsection heading] Trafficking is characterized by different recruit- 
ment methods which [ABE_Business] 


[28] Marketing ethics and consumer behavior [= section heading] Marketing eth- 
ics refers to the moral principles behind the operation [ABE_Marketing] 


[29] possible identity shifts. One group of the returned expatriates is classified as 
“identity shifters” which can be explained through their exposure [ABE_Eco- 
nomics] 


[30] consumer manipulation in at least four domains. The first field refers to food 
products, where, for instance, manufacturers [ABE_Business] 


The two modes of expressing technicality are also consistent with regard 
to the following usage pattern: 


[31] Ishall briefly describe the idea of fair trade. Then I shall explain and discuss 
the impact of the social, economic [ABE_Business] 


[32] CSR measures at COMPANY. In a first step, I shall explain what drives 
companies to engage in CSR [ABE_Business] 


Just like the verb to define, to explain is not used as a reporting verb either. 
Instead, it tends to function as a gloss in the strict sense of a metatextual 
comment (witness the co-occurrence with frame markers). Once more, the 
business students are certainly aware of genre conventions and assessment 
criteria, which, as mentioned in Section 3.1, include presenting an academic 
argument on the basis of reasoning. 


[33] characteristic is the oligopoly market situation which means that the market 
is dominated by a few companies that fight over market [ABE_Economics] 


[34] developing countries and has not changed after independence. Furthermore, 
these countries are characterized by top-down management with “authoritar- 
ian and paternalistic [ABE_Business] 


As these examples indicate, students have their ways of responding to the 
requirements laid down for SP writing. They demonstrate their understand- 
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ing of the concept ‘oligopoly market situation’ by providing an explanation 
using their own words rather than relying on attribution. It remains to be 
seen, however, whether or not they do so as a result of the nature of the SP 
“in which writers want to show their instructor what they have read (and 
hence what they know), but what they think is of minor importance for 
their paper” (Gruber 2004: 54). While this compliance may well be linked 
to the SP’s main purpose of accreditation, it may also point to the absence 
of more assertive writer identities (Sing 2015). Alternatively, both factors 
could be related to the institutional setting and viewed as the outcome of 
localized forms of learning. Prior to addressing this important point, I will 
briefly wrap up the discussion of the results. 

Centring on the notion of technicality and relating it to Hyland’s (2005) 
interpersonal model of academic writing, the present study conceived of 
technicalizing as a two-stage process, involving the naming of the phenom- 
enon concerned as well as integrating it into a field of knowledge. The cor- 
pus analysis revealed that student writers use a number of metadiscoursal 
devices, so-called code glosses, in order to flag the technical nature of their 
writing. While it is understood that writers technicalize in a variety of ways 
and using a range of linguistic resources other than the ones examined here, 
the corpus analysis succeeded in retrieving highly relevant usage patterns 
that may be fed back into teaching. Cases in point are the different uses and 
functions of the chains of reference established by the student writers when 
elaborating, or trying to elaborate, on new terminology. The addition trying 
to is especially noteworthy as the preceding discussion may have suggested 
that the usage patterns identified are taken at face value. Whether or not 
the student writers intend to define, explain or exemplify when using the 
respective metadiscoursal devices is a moot point. Owing to the explora- 
tive character of the present study, the issue of congruence is an important 
corollary of the analysis. Further empirical work, possibly involving focus 
group interviews with the students concerned, is required to establish if the 
discourse action and the discourse label used to denote the action can, with 
some justification, be argued to converge. 
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5 Conclusions: Meeting conceptual challenges 


At the outset, the article invoked the challenges of contemporary ESP set- 
tings in view of the major changes in HE across Europe. Many of the 
challenges - linguistic, cultural and conceptual - have regularly dominated 
specific-purpose instruction. This ongoing process can be accounted for by 
a widening of the gap between ESP research and practice. To date, teaching 
and materials development are surprisingly little informed by state-of-the- 
art research, even in those cases which would lend themselves to direct 
teaching applications. 

What is more, needs analysis retains a strong foothold in more tradi- 
tionally inclined ESP settings, assuming that once the area and nature of 
students’ difficulties are identified, they can be more easily assisted in mas- 
tering the language-based skills or tasks required. In view of the growing 
complexity of HE, ESP programmes are, however, frequently forced to cater 
to vastly different literacy demands, spanning the workplace, academia 
and the corporate world. What is needed is a stronger commitment to the 
relevance of context in ESP, including an acknowledgement of the situated- 
ness of learning. 

Using the field knowledge of an ethnographic researcher, I will now turn 
to discussing several pedagogical implications for ESP writing instruction. 
The inconsistency between language and content typical of ESP settings 
becomes manifest in the writing task as the distinction between ‘carrier 
content’ and ‘real content’, which is central to ESP. In specific-purpose 
instruction, teaching activities are essentially context-based, implying that 
content knowledge is used to teach and practise specific language. In the 
ESP setting at hand, the business topics of the SP are not the aim of the 
task; rather, these topics constitute the carrier content for which written 
academic English is the real content, i.e., the explicit teaching target. This 
is, however, not the only reason why the SP is a genuine showcase for ad- 
dressing conceptual challenges in ESP writing instruction. 

What is more, students seem to struggle with the writing task itself. 
Given the absence of any formal writing instruction prior to producing the 
assignment, students come equipped with rather common (mis-)conceptions 
about academic writing, regarding it as a monolithic entity. Setting the SP 
as a simple assignment further misconstrues writing as a product. Instead, 
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it would be useful to foreground the processual character of writing, for 
example, in setting up a compound assignment, in which individual subgen- 
res are trained separately. In this manner, the different resources that can 
be used to technicalize in writing could be singled out, paying particular 
attention to structural issues such as composition taxonomy. 

It is on the level of categorization rather than language that students are 
in need of assistance. Despite imperfections in grammar and style, students’ 
actual conceptual problem lies in taxomizing. As the findings suggest, stu- 
dents rely heavily on the structure already provided to them in the sources 
they use. Only few student writers have managed to reorganize the mater- 
ial covered in an original way. In view of the patterns identified, the large 
majority of students seem to address this requirement through retaining the 
superordination taxonomy. Bold as it may seem, such an assertion is fully 
supported by the contextual information available to the compiler-cum- 
analyst of the ABE corpus. On the basis of this inside knowledge, such a 
student response is hardly surprising. On the contrary, it is a conditioned 
reflex acquired in a learning environment that rewards learning strategies 
such as rote learning, which is then simply transferred to writing. The busi- 
ness students at hand emulate what they consider expert models and seek 
to apply these to their own writing. 

This begs the question as to what type of expertise these students are 
aspiring to. Being susceptible to a technocratic view of the experiential 
world, it seems likely that these expert genres are realized by text types 
that are dramatically different from the SP. Thus the discrepancy between 
the educational genre, the training text used for writing practice and the 
professional genres used in the workplace could not be more apparent. If 
assignment writing requires students to “fictionalize” (Pohl 2009), this is 
particularly valid for this ESP setting. Using the SP with its cultural bag- 
gage as a training text only serves to alienate students from the professional 
target genres. With a view to developing their writing skills as business 
students, these professional genres need to be firmly anchored in the rel- 
evant curricula. Although policy makers pay lip service to the importance 
of language learning, formal writing instruction in particular has been in- 
creasingly marginalized in curricular development. It seems to be the case 
that specific-purpose instruction is clearly seen as training rather than an 


integral part of language education. 
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Abstract 

English: This paper is centred in the context of English Medium Instruction (EMI) 
and is primarily concerned with advanced students’ productive knowledge of Eng- 
lish academic vocabulary, widely regarded as a crucial dimension of successful aca- 
demic communication. The study problematizes the claim that EMI is beneficial for 
students’ development of academic vocabulary knowledge. The investigative context 
is a technical university in Sweden where all degree programmes at graduate level use 
English as the medium of instruction. The corpus data include texts (n=80, approx. 
720,000 words) produced by Master of Science students in their first and second 
year of study, written by home and international students. The study, using the 
Academic Vocabulary List (Gardner/Davis 2014), sets out to answer three research 
questions relating to knowledge and development of academic vocabulary in EMI: 
1. What is the lexical coverage of advanced (master’s) level student writing, i.e., 
what proportion of words in students’ texts is academic? 2. Are home students and 
international students (all of whom have English as a foreign language) comparable 
in terms of their productive academic vocabulary knowledge? 3. Does students’ 
productive knowledge of academic words appear to develop during their studies? 
The results of the investigation can be summarized as follows: In the corpus as a 
whole, academic vocabulary items account for approximately 20% of all tokens. 
This figure is considerably higher than that found in many earlier studies. There 
are no significant differences between home and international students in any of 
the measures of vocabulary used (pertaining to lexical sophistication and diver- 
sity). Finally, the findings regarding lexical development across years of study are 
somewhat mixed; however, the overall picture presented by the various measures 
is one of significant but very modest gains in some areas and none in others. These 
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findings call into question the actual effectiveness of EMI for academic vocabulary 
development. The overall contribution of the paper is an important step towards a 
more comprehensive understanding of what expectations we may reasonably have 
of the development of English language competency in EMI. 


German: Die Fahigkeit, wissenschaftlichen Wortschatz aktiv zu gebrauchen, gilt 
als eine entscheidende Komponente erfolgreicher Wissenschaftskommunikation. 
Dabei herrscht die Annahme vor, dass English Medium Instruction (EMI) einen 
positiven Einfluss auf die Entwicklung des wissenschaftlichen Wortschatzes Studie- 
render habe. Im vorliegenden Beitrag werden die Ergebnisse einer Studie vorgestellt, 
die diese Annahme kritisch beleuchtet. Die Studie wurde an einer Technischen Uni- 
versität in Schweden durchgeführt, an der alle Masterstudiengänge auf Englisch 
unterrichtet werden. Das Korpus umfasst Texte (n=80, ca. 720.000 Wörter), die 
von schwedischen und internationalen Master-of-Science-Studierenden in ihrem 
ersten und zweiten Studienjahr verfasst wurden. Die Studie, für die die Academic 
Vocabulary List (Gardner/Davis 2014) genutzt wurde, geht drei Forschungsfragen 
nach, in deren Mittelpunkt der Umfang und die Entwicklung wissenschaftichen 
Wortschatzes in EMI-Kontexten stehen: 1. In welchem Umfang beherrschen fort- 
geschrittene Masterstudierende den wissenschaftlichen Wortschatz, d. h., welcher 
Anteil der Wörter in den studentischen Texten ist wissenschaftlich? 2. Sind schwe- 
dische und internationale Studierende mit Englisch als L2 vergleichbar in ihrer 
Kompetenz, wissenschaftlichen Wortschatz aktiv zu gebrauchen? 3. Entwickelt sich 
dieser Wortschatz während des Studiums erkennbar weiter? Die Ergebnisse der Un- 
tersuchung lassen sich wie folgt zusammenfassen: Im Korpus hat wissenschaftliches 
Vokabular einen Anteil von ungefähr 20 % aller Tokens und damit einen deutlich 
höheren Anteil, als in vielen früheren Studien nachgewiesen werden konnte. Im 
Gebrauch wissenschaftlichen Wortschatzes (sowohl hinsichtlich dessen lexikalischen 
Anspruchs als auch dessen Differenziertheit) lassen sich zwischen schwedischen 
und internationalen Studierenden keine signifikanten Unterschiede feststellen. Die 
Befunde zur Entwicklung des Wortschatzes während des Untersuchungszeitraums 
sind ambivalent; insgesamt lässt sich jedoch ein moderater Zugewinn in einigen Be- 
reichen feststellen, wohingegen in anderen Bereichen kein Fortschritt zu verzeichnen 
ist. Diese Ergebnisse geben Anlass zu berechtigten Zweifeln an der Annahme, dass 
sich mit EMI Wortschatz effizient erweitern lasse. Die Studie leistet somit einen 
wichtigen Beitrag zur realistischeren Einschätzung der Möglichkeiten, die EMI für 
die Förderung der englischen Sprachkompetenz bietet. 
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1 Introduction 


The broader context of this study of academic vocabulary knowledge is 
English Medium Instruction (EMI). For the purposes of this paper, this may 
be defined as the deliberate use of English (typically as a result of an official 
educational policy) to engage students communicatively in academic study, 
i.e., by asking students whose first language is not English to read, write, 
speak and listen in English rather than using their first language (cf. Cole- 
man 2006; Dearden 2014). While in some contexts a distinction between 
EMI and Content and Language Integrated Learning (CLIL) is becoming 
increasingly difficult to maintain, EMI and CLIL should not be considered 
the same thing. The difference might be one of degree but the defining fac- 
tor distinguishing the two is the extent to which students receive deliberate 
language education (CLIL) as opposed to mere immersion (EMI) (Marsh 
2005; Lasagabaster 2008; Gustafsson et al. 2011; Gustafsson/Jacobs 2013). 

EMI is a “rapidly growing global phenomenon” (Dearden 2014: 2) and 
a number of different “drivers of the Englishization” (Coleman 2006: 4) 
of higher education exist. In this respect, there is a widespread assumption 
articulated to differing extents by the various stakeholders that students 
enhance their academic as well as general English competency as a result 
of studying in EMI contexts. However, this is only an assumption (like 
many other assumptions about EMI reviewed by Dearden), and it has yet 
to be confirmed by empirical research. Dearden (2014: 2) highlights the 
fact that “we are quite some way from a ‘global’ understanding” of EMI 
and notes that there is an “urgent need for a research-driven approach [...] 
which measures the complex processes involved in EMI”, for example, the 
conditions for “the acquisition of English proficiency”. In other words, re- 
search that confirms, refutes, or at least problematizes the claim that EMI is 
beneficial for students’ development of English language skills is called for. 

In this paper, we are concerned with a single but crucial dimension of 
English proficiency development in EMI contexts, namely academic vo- 
cabulary knowledge, and, more specifically, students’ productive knowledge 
of academic words as reflected in their writing. Our starting point is the 
widely held claim (see, e.g., Stachr 2008) that there is a correlation between 
knowing many words, i.e., having good vocabulary knowledge, and overall 
communicative competence. Milton (2010: 212) notes that “vocabulary 
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knowledge is key to both comprehension and communicative ability”, and 
Laufer & Nation (1999: 34) talk of the “enabling” function of vocabulary 
vis-a-vis other dimensions of communication. In academic discourse, the 
same kind of correlation obtains between academic words and general 
academic literacy (see, e.g., Corson 1997; Coxhead 2000; Milton 2010). 
These correlations are supported by research indicating that understanding 
virtually all the input words is fundamental to comprehension in any kind 
of communicative situation (Coady/Huckin 1997; Schmitt 2000; Nation 
2001; Bogaards/Laufer 2004). It has been suggested that in excess of 95% 
of the running words must be understood for “adequate comprehension” to 
be possible in connection with reading and listening, and for optimal com- 
prehension as much as 98% of the words should be known (Nation 2001; 
2006). It seems evident that, if a large vocabulary is needed for reading 
comprehension, it must be of at least equal importance for the productive 
assessment tasks in an EMI environment. 

In this respect, it is reasonable to ask what might actually be expected 
of students in EMI in terms of academic vocabulary knowledge. Numerous 
methods of measuring vocabulary knowledge exist, including self-assess- 
ment scales and definition tasks among other measures of receptive knowl- 
edge. However, in most EMI settings, students need not only to understand 
English in lectures, textbooks, etc., but also to produce it in assessment 
tasks, so their productive vocabulary knowledge is of interest. Our first 
research question is therefore: 


1. What is the lexical coverage of academic vocabulary in student writing, i.e., 
what proportion of words in students’ texts is academic? 


A second perspective we want to explore concerning students’ produc- 
tive knowledge of academic words relates to the Englishization of higher 
education as a result of globalization/student mobility. One effect of glo- 
balization and the concomitant proliferation of EMI has been a rise in 
EMI outside of the traditionally English-speaking world, in the “Expanding 
Circle” (Kachru 1992).? In Sweden, where this study is set, as in many other 


2 Kachru (1992) identifies three English “Circles”. The Inner Circle is repre- 
sented by countries like the United Kingdom, USA, Canada, Australia, and New 
Zeeland, i.e., countries where English is the primary language and the native 
language of most people. In Outer Circle countries (such as Singapore, India, 
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Expanding Circle countries, a result has been that significant numbers of 
‘international’ students (students domiciled outside Sweden) are now study- 
ing alongside ‘home’ students (domiciled in Sweden with an almost 100% 
Swedish language background and English as their first foreign language at 
school), creating an international mix and a multilingual learning environ- 
ment in what used to be a linguistically relatively homogenous teaching/ 
learning environment. The (English) language demands placed on interna- 
tional students and home students in the EMI classroom are naturally the 
same, but our overall knowledge of how this more diverse group of students 
actually performs linguistically vis-a-vis home students is very limited. 

It is easy to problematize international students and find isolated and cat- 
egorical statements that speak in general negative terms about international 
students’ shortcomings with regard to English proficiency. For example, in 
response to a survey about attitudes towards English in higher education 
administered to university teachers in Sweden (reported in Pecorari et al. 
2011), one teacher offered the following comment: 

“English texts, especially academic English texts that we use, are demanding for 

students. The same is true for writing in English. The problem is especially pro- 


nounced for our foreign students who are particularly challenged to write accept- 
able English.” 


However, with very few exceptions (see, e.g., Jochems et al. 1996), the re- 
search available concerning the overall academic performance and linguistic 
ability of international students in relation to home students is restricted 
to Inner Circle countries (see, e.g., Warwick 2006; Carroll/Ryan 2005; 
Morrison et al. 2005), meaning that our knowledge of how international 
students compare to home students in Expanding- and Outer Circle coun- 
tries is almost non-existent. 

A first step towards a more comprehensive understanding of the rela- 
tionship between home students and international students as regards their 


Pakistan, Bangladesh, Nigeria, and Kenya) English is not the native language but 
firmly established as the lingua franca in most areas of society and typically has 
the status of “official language”. Finally, in countries in the Expanding Circle 
(the Nordic countries are a case in point), although often widely used for the 
purpose of international communication (e.g., in much business communica- 
tion), English is not an official language or the language used in government. 

Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 

Downloaded from PubFactory at 01/11/2019 10:15:23AM 

via free access 


50 H. Malmström / D. Pecorari / M. Gustafsson 


English proficiency is to establish whether home students and international 
students in EMI environments have equally sized academic English vo- 
cabularies, clearly a pertinent investigation given the centrality of academic 
words to comprehension. This leads us to pose the second of our three 
research questions: 


2. Are home students and international students (all L2 users of English) compa- 
rable in terms of their productive vocabulary knowledge? 


Finally, this paper directly addresses the supposition that EMI is conducive 
to developing students’ English language skills, focusing on vocabulary 
gains. There is support in the literature that incidental exposure to English 
vocabulary in study contexts does lead to positive lexical gains over time 
(Huckin/Coady 1997; Laufer/Hulstijn 2001); however, the vast majority of 
this research has focused on learners’ receptive knowledge of vocabulary 
and there is a dearth of research concerned with students’ productive lexi- 
cal knowledge (cf. Durrant 2014). In addition, the bulk of the literature on 
vocabulary acquisition, and indeed second-language acquisition generally, 
has focused on learners who are much less advanced than those who are in 
a position to undertake study at university through the medium of their L2. 

Laufer (1994), the most widely cited study on productive vocabulary 
development available, looked at “changes in the productive lexicon of 
advanced second language learners’ writing over a period of one academic 
year” (1994: 21) using a construct she calls “lexical quality”. Focusing on 
writing compositions produced in a controlled environment, and by draw- 
ing on two basic types of analytical measures, a frequency profile and a 
Type-Token Ratio, Laufer found no significant lexical gains with regard to 
general high-frequency words, but there were significant gains for words 
from the University Word List (Xue/Nation 1984), i.e., Laufer’s measure 
used for academic words, and for words of lower frequency (words beyond 
the 2,000 most common). With respect to lexical variation, as measured 
by the Type-Token Ratio, no significant longitudinal gains were recorded 
for any type of lexis. 

Additional longitudinal perspectives on the development of productive 
lexical knowledge have been provided by more recent research from Aus- 
tralia. This research comes to a different conclusion. Knoch et al. (2015) 
investigated to what extent the writing of English L2 students developed 
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positively over three years. Development was measured by looking at a set 
of discourse-analytic measures, among which was lexical complexity, op- 
erationalized with reference to the proportion of words from the Academic 
Word List (Coxhead 2000), lexical sophistication and lexical richness. For 
all three measures of lexical complexity, the differences between the first 
and the second writing collection point fell well short of statistical signifi- 
cance, suggesting that there is little support for the notion that studying in 
an EMI context has a significantly positive effect on students’ productive 
knowledge of academic lexis. 

The setting provided by Laufer’s study as well as the study reported by 
Knoch et al. is in many ways different from EMI contexts in Europe out- 
side Great Britain and Ireland and elsewhere today. It is noteworthy, for 
example, that the participants in Laufer’s investigation were all language 
learners and therefore possibly ‘primed’ to attend to linguistic matters like 
vocabulary, unlike the vast majority of EMI students enrolled in subject 
courses or degree programmes where there is little or no attention devoted 
to language per se. In addition, it seems unfair to compare the EMI situation 
of English L2 students studying in English Inner Circle countries with the 
situation in Expanding- or Outer Circle countries; the complete immersion 
in an Inner Circle environment presumably affords many more opportuni- 
ties for engagement with English vocabulary (academic or otherwise). The 
issue of lexical development in the EMI context facing a great number of 
students in Expanding- or Outer Circle countries must therefore be inves- 
tigated independently of such research in Inner Circle countries. Thus, the 
third research question that this study asks is: 


3. Does students’ productive knowledge of academic words appear to develop 
during their studies in an EMI context? 


2 Data collection and methods 


This section describes the context of the investigation, the data collection 
and the analytic procedures adopted. 
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2.1 Study context 


Our study is set at a prestigious technical university in Sweden where policy 
stipulates that all degree programmes at master’s level use English as the 
medium of instruction. The university has approximately 11,000 students, 
2,400 of whom are enrolled in one of 41 master’s programmes. While we 
have no reliable record of the first language of the students, the majority of 
the home students are Swedish L1 speakers and, as far as we have been able 
to ascertain, none of the international students’ whose data were included 
were domiciled in an English L1 country. 


2.2 Data 


The data for this study is a small student text corpus made up of 80 texts 
in English (totalling just over 720,000 running words) written by Master 
of Science (MSc) students from four different disciplines (applied physics, 
chemical engineering, chemistry, and mechanical engineering) and from the 
first and second year of study at the master’s level. A total of 30 texts, com- 
prising approximately 115,000 running words, were primarily technical- or 
mini-project reports written as part of students’ course work at some point 
during the first year. The 50 second-year texts, comprising approximately 
605,000 words, were full-length master’s theses written during the last 
term of a two-year study programme. At this university, master’s theses are 
generally reports of project work. Thus, although there is greater variation 
in the first-year corpus in terms of the assignment set, the two sub-corpora 
can be regarded as broadly similar in terms of text type. 

Because virtually all written course work at this university is done in 
groups of two or more students, we were unable to obtain first-year and 
second-year texts from the same author or team of authors. Therefore, 
‘development’ of academic vocabulary refers to change across levels of 
study rather than change in individual students. In all cases, all authors 
were either Swedish or international; texts with a mixed authorship with 
regard to national origin were excluded from the sample in order to enable 
the comparison between Swedish and international MSc students regarding 
their productive knowledge of English academic vocabulary. 
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2.3 Analytical procedure 


To address the three research questions, we used several measurements, 
following Milton (2009) in distinguishing between two basic constructs: 
lexical sophistication and lexical diversity. The former refers to the extent 
to which more or less common words are used: in Milton’s example, the 
difference between the cat sat on the mat and the feline reposed on the 
antique Persian rug (2009: 131). The latter refers to the extent to which 
the same or different words are used. 

One measure of lexical sophistication is the presence of academic vocabu- 
lary. Over the years, several descriptions and compilations of academic vo- 
cabulary have been developed (see Gardner/Davies 2014; and Charles/Pecorari 
2016: 109 ff. for a discussion). Until recently, the most widely used list, for 
both teaching and research, has been Coxhead’s (2000) Academic Word List 
(AWL). As a result of this wide use, a number of limitations of the AWL have 
been identified,’ and these limitations have provided the impetus for the newer 
Academic Vocabulary List (Gardner/Davies 2014); this list is adopted as a 
basis for what counts as academic vocabulary in the present study.* 

The AVL was developed from a 120-million-word academic sub-corpus 
(featuring texts with a heavy emphasis on journal articles from across nine 
different academic disciplines) taken from the 425-million-word Corpus 
of Contemporary American English (COCA 2015). Rather than using an 
existing word list to exclude general vocabulary (the way Coxhead used 
the General Service List by West 1953), a criterion for relative frequencies 
was developed, such that words were considered to be part of an academic 
core if they occurred in the academic corpus with a frequency 50% greater 
than in the general, non-academic reference corpus (the non-academic por- 
tion of COCA). Words needed to be represented at or above a threshold 


3 It is not within the scope of this article to criticize the AWL, but Hyland & Tse 
(2007) and Gardner & Davies (2014) both offer a comprehensive account of 
the perceived problems with the AWL. 

4 The pedagogic utility of lists of general academic vocabulary is widely accepted 
and Gardner & Davies (2014: 2) note several areas in which such lists are 
purposeful (see also Schmitt/Schmitt 2014). It should be stressed, however, that 
pedagogic utility is not a central concern in this study (though see Section 6 
where various didactic implications are discussed). 
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frequency in seven of the nine subject areas, and both a criterion for disper- 
sion and relative frequencies were implemented to exclude words which 
have a particular affinity with one or a few subject areas (thus allowing for 
a distinction to be made between core/general academic vocabulary and 
subject-specific/technical vocabulary). As a result of this process, the AVL 
consists of 3,015 words (lemmas) that occur across a wide range of aca- 
demic disciplinary areas more frequently than they do in general discourse. 
Table 1 includes examples of words from the AVL. 


Table 1: Most and least frequent words (lemmas) in the Academic Vocabulary List 


AVL words 1-10 AVL words 3006-3015 
study n. however adv. unusable adj. imprimatur n. 
group n. research n. unpalatable. adj. | coherently adv. 
system n. level n. causally adv. component n. 
social adj. result n. prioritization n. tangential adj. 
provide v. include v. overemphasis n. relevancy n. 


Because the AVL contains words that are more common in academic than 
general discourse, they are in that sense ‘advanced’ vocabulary. Two meas- 
ures of lexical sophistication used in the present study were based on the 
AVL: the proportion of coverage afforded by the AVL, and the number of 
types from the AVL. However, AVL items vary greatly in frequency, and 
so not all are equally ‘advanced’. For example, the most frequent word 
on the AVL is study, and it also is among the first 1,000 general words by 
frequency. We thus distinguish between the first 500 words on the AVL 
and the rest of the list. 

The second construct we were interested in was lexical diversity. One of 
the oldest (Johnson 1939; Mann 1944) and most common measures of lexical 
diversity is the Type-Token Ratio (TTR). However, the TTR has limitations 
(Malvern/Richards 2013; Vermeer 2000), including the fact that it is sensitive 
to text length (Holmes 1994; Baker 2006). While there are no entirely unprob- 
lematic measures of lexical diversity, the Guiraud Index (Guiraud 1954) and 
the Advanced Guiraud (Daller et al. 2003) compensate for the TTR’s sensi- 
tivity to length and perform more reliably, and were thus adopted here. The 
former is calculated by dividing the number of types in a text by the square 
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root of the number of tokens, and the latter uses the same calculation after 
very common words have been excluded (in this case, the first thousand most 
frequent words in the BNC and COCA corpora, as provided by Paul Nation’s 
Range files (Range 2015)). Because the advanced measure eliminates the most 
frequent types, there is a basis for considering it to be an indicator of lexical 
sophistication as well as diversity, as Daller & Xue (2009) do. Because there 
is no established baseline for these measures in texts of the type analysed here, 
they are primarily of value in this study in the two between-group comparisons. 

Once the data had been collected, the texts were cleaned’, converted into 
text files and processed using AntWordProfiler (Anthony 2015) to deter- 
mine the frequency of AVL words. In vocabulary profiling (see, e.g., Laufer 
1994; Nation 2006) the number of words (tokens) in the texts is counted 
and the words’ distribution relative to pre-established lists is calculated. 
In this case we used two lists: one list consisting of the 500 most frequent 
types in the AVL (called AVL 500 here), and one comprising the remaining 
less frequent lemmas from the AVL (AVL 501+). 

Two additional procedures were needed to enable a comparison of the 
present findings with the Gardner & Davies (2014) study (to the best of our 
knowledge, the only study based on the AVL to date). Unlike the COCA 
academic sub-corpus used in that study, our corpus is untagged, meaning 
that it does not distinguish between words like study, n., which is on the 
AVL, and study, v., which is not. To estimate the effect of this difference, 
a manual search was done among the first 300 words of the AVL for can- 
didates for overcounting (such as study, v.). A second procedural issue is 
that the profiling tools used in the study may have idiosyncrasies which 
cause them to perform somewhat differently. To estimate the extent of this 
effect, samples of each corpus text were submitted individually to the lexi- 
cal profiling tool on Mark Davies’ Word and Phrase website (Davies 2015) 
which analyses the first 1,000 words from each text. 


5 The following features were removed from the texts: extensive visual informa- 
tion in the form of tables and figures (table and figure captions were left in); 
all equations/formulae and/or parts thereof, unless some element featured as a 
syntactic constituent in which case it was treated as technical vocabulary; finally, 
all tables of contents, reference sections and acknowledgement sections were 
also removed. 
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Finally, where relevant, SPSS was used to test the significance of between- 
group differences. Because a random distribution could not be assumed, 
a non-parametric test was appropriate (Turner 2014). The independent 
samples Kruskal-Wallis test was used, and differences were considered sig- 
nificant when p < .05. 


3 Results 


The research questions guiding this investigation related to 1) academic 
vocabulary coverage; 2) comparisons between home and international stu- 
dents; and 3) comparisons between first-year and second-year texts. 

The results of the AVL profiling (Fig. 1), relating to the lexical sophis- 
tication of the texts, showed that 19.3% of the tokens in the corpus are 
academic words. This is a considerably higher proportion of academic 
vocabulary than previous studies have shown. The most relevant earlier 
study is Gardner & Davies (2014), who found that the AVL gave coverage 
of the academic sections of the COCA and BNC in the vicinity of 14%. 
To estimate the extent to which procedures may have contributed to these 
different results, two additional analyses were conducted. 


AVL Coverage 


E General / 
Technical 


AVL 


Fig. 1: Academic vocabulary coverage in the MSc writing corpus 


To account for the effect of the untagged corpus, a manual search was done 
among the first 300 words ofthe AVL for candidates for overcounting (such 
as study, v.). A total of 2,028 such forms were identified, or approximately 
1.4% of the total corpus size. The effect of the untagged corpus is therefore 
real but relatively modest. When the first 1,000 words of each text were 
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submitted to the Word and Phrase profiling tool (Davies 2015), an aver- 
age of 23% of the tokens came from the AVL. It is therefore reasonable to 
conclude that academic words do in fact make up approximately 20% of 
our corpus and that procedural issues play a relatively small role. Other 
explanations for the difference between these findings and earlier studies 
are taken up in Section 4 below. 

A further measure of lexical sophistication was the proportion of infre- 
quent AVL words. In the COCA academic corpus, the 500 most frequent 
of the 3,015 AVL types (i.e., 17%) account for 74% of all of AVL tokens. 
The average number of tokens representing each type in the 1-500 list was 
14 times greater than on the 501+ list (22,599 versus 1,480) (see Fig. 2). As 
Figure 2 shows, the figures for the present corpus are comparable: 70.5% of 
the AVL tokens come from the first 500 words of the AVL, with only 29.5% 
coming from the remainder of the list, and the average type on the 1-500 
list had nearly 13 times as many tokens as the average type on the 501+ list. 


AVL Coverage Distribution AVL Coverage Distribution 
(COCA) (MSc) 
m AVL500 E AVL500 
AVL501+ AVL501+ 


AVL distribution COCA 


AVL distribution MSc 


1480 
E AVL500 E AVL500 
type type 
AVL501+ AVL501+ 
type type 


Fig. 2: Academic vocabulary diversity and sophistication in the COCA and MSc 


writing corpora 
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The second research question addressed the relative productive vocabulary 
of native Swedish and international students. Given that large numbers of 
international students are a relatively recent phenomenon in Swedish uni- 
versity classrooms, there is a need to understand whether the English skills 
of this new constituency permit them to participate in EMI on the same 
terms as their local counterparts. Measures for lexical sophistication and 
lexical diversity were therefore considered for the two groups separately. 

As Table 2 shows, for all measures, the differences were small, and in- 
deed none of them was statistically significant. This measure thus suggests, 
encouragingly, that this relatively new student group is able to take on EMI 
education on a level playing field with their Swedish peers, at least when 
assessed on the basis of the productive vocabulary knowledge. 


Table 2: AVL distribution for home versus international students 


Measure Overall Home International 
AVL coverage (entire list) 19.3% 18.9% 19.9% 
AVL 500 coverage 13.6% 13.2% 14.1% 
AVL 501+ coverage 5.74% 5.7% 5.8% 
AVL Types per 100 words 5.69 5.88 5.40 
AVL 500 types per 100 words 3.60 3.76 3.37 
AVL 501+ types per 100 words 2.08 2.12 2.03 
Guiraud Index 14.2 14.01 14.48 
Advanced Guiraud 8.68 8.33 9.20 


Since language development is one of the reasons offered for implementing 
EMI, it would be reasonable to think that students’ vocabulary — particu- 
larly academic vocabulary — develops during their course of study. The 
third research question was therefore whether the second-year texts showed 
greater lexical sophistication and variation than the first-year texts. 

The measures of lexical sophistication failed to reflect gains between the 
two groups. As Table 3 shows, a small increase was found for the overall 
coverage afforded by the AVL, from 19.0% to 19.5%. On closer investiga- 
tion, this increase is seen to be driven by increased usage of the 500 most 
frequent AVL items, from 12.9% to 13.9%, which was in turn offset by a 
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slight decrease in coverage from the remainder of the list. However, none 
of these changes were significant. 

The normalized frequencies of types from the entire AVL, the most fre- 
quent 500 items and the less frequent items, all showed a significant de- 
crease in the second year (p=.000). This (in combination with the change in 
the Guiraud Index and Advanced Guiraud, see below) is an indication that 
the greater diversity of lexis came either from general (i.e., non-academic) 
vocabulary in the less frequent range, and/or from technical terminology, 
rather than increased usage of general academic vocabulary. 

In terms of variation, both the Guiraud Index and the Advanced Guiraud 
showed a modest but significant (p=.000) trend toward greater lexical 
variation in the year-two texts (see Table 3). Because it excludes the first 


thousand most commonly used words, the Advanced Guiraud also reflects 
lexical sophistication. 


Table 3: AVL distribution in first- versus second-year texts 


Measure Overall Year 1 Year 2 
AVL coverage (entire list) 19.3% 19.0% 19.5% 
AVL 500 coverage 13.6% 12.9% 13.9% 
AVL 501+ coverage 5.74% 6.0% 5.6% 
AVL Types per 100 words 5.69 7.40 4.73 
AVL 500 types per 100 words 3.60 4.85 2.90 
AVL 501+ types per 100 words 2.08 2.55 1.82 
Guiraud Index 14.2 12.96 14.90 
Advanced Guiraud 8.68 7.46 9.37 


4 Discussion 


Section 3 presented findings that were in some ways unexpected, and thus 
merit further discussion. 


4.1 Academic vocabulary coverage 


The finding that approximately 20% of this corpus consisted of academic 
words contrasts strikingly with the much lower figures in previous stud- 
ies. For example, Coxhead (2000) and Hyland & Tse (2007) found that 
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the Academic Word List afforded about 10% coverage of their respective 
corpora. A 20%-figure was approached only in Chung & Nation’s (2003) 
study concerning Applied Linguistics textbooks. As noted above, method- 
ology accounts for only a minor part of the difference, and it is interesting 
to speculate as to what may account for the rest. 

One likely explanation is the composition of the corpus; the propor- 
tion of academic vocabulary varies according to text type (Chung/Nation 
2003; Li/Qian 2010) and academic discipline (Chung/Nation 2003; Cox- 
head 2000; Hyland/Tse 2007). However, no other study based on a fully 
comparable corpus exists. Engineering was one of the fields investigated 
by Hyland & Tse (2007) and Mudraya (2006), but the former corpus 
contained a mix of text types, while the latter consisted of textbooks and 
provided no overall academic vocabulary coverage figure. Thus, while it 
is probable that academic subject area and text type explain some of the 
difference between the present findings and earlier ones, it is not possible 
to ascertain the extent of their influence. 

A second explanation lies in the use here of the Gardner & Davies (2014) 
AVL, while previous investigations have employed Coxhead’s (2000) AWL. 
The AVLs ability to represent core academic vocabulary better than the 
AWL has been demonstrated empirically. Gardner & Davies (2014) profiled 
the academic sections of the BNC and COCA with both the AWL and the 
AVL, using word families to enable a comparison with the AWL, and found 
that the top AVL 570 word families provided nearly twice the coverage with 
respect to the AWL (13.8% versus 7.2% in COCA; 13.7% versus 6.9% in 
the BNC). In this light, the fact that the present study found approximately 
twice as much academic vocabulary as earlier studies is unsurprising; indeed 
an investigation of the present MSc writing corpus (with some modifica- 
tions) found that the AWL provided just under 10% coverage (Gustafsson/ 
Malmström 2013). 

A further question is why AVL coverage is higher for the MSc writing 
corpus than the academic portions of COCA and the BNC. Here too, 
corpus composition undoubtedly plays a role. In addition, there is likely 
to be an effect due to an aspect of Gardner & Davies’ (2014) methodol- 
ogy. While the AVL (unlike the AWL) is not based on word families, their 
figure of approximately 14% coverage comes from a case study which, in 
order to permit comparisons with the AWL, used part of the AVL grouped 
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into 570 word families. The findings of these studies are therefore not 
fully comparable, and because of the lack to date of studies using the AVL, 
further research is needed. 


4.2 Home versus international students 


There is a fairly commonplace belief on the part of many Swedish uni- 
versity teachers that international students have lower English proficiency 
than home students. This perception is frequently offered almost apologeti- 
cally; there is a widespread perception that international students enrich the 
Swedish university classroom and that their presence is therefore desirable, 
but that achieving an international student presence requires the use of 
English as an academic lingua franca, and while this puts all participants 
at a disadvantage, those who have gone through the Swedish educational 
system, which emphasizes English, are better able to cope than most incom- 
ing mobile international students. It is not clear how to explain the disparity 
between this belief and the findings of the present study. 

A possible explanation is that teacher perceptions are based less on real- 
ity and more on an awareness of differences. More specifically, the English 
used by Swedish university students is familiar to their teachers, and the 
non-standard transfer features that characterize it are unmarked, while 
those of students with other origins are more salient. Another possibility, 
which indeed is applicable to all of the findings reported here, is that stu- 
dents recruited to the prestigious university where this study was conducted 
are a relatively homogeneous, skilled group of English users. Were the study 
to be replicated at another institution, between-group differences might be 
identified. It is also possible that these groups may differ in English profi- 
ciency, but that the differences manifest themselves in other domains than 
productive academic vocabulary (i.e., in other domains of oral and written 
communication). Future research would be required to establish the extent 
to which any of these explanations is a factor. 


4.3 Vocabulary development over time 


One of the intended benefits of EMI is that it creates exposure to the 
language and can therefore result in incidental vocabulary acquisition. 
Academic vocabulary would appear to be a prime candidate for such ac- 
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quisition, since it is an area of language to which students can be expected 
to have greatest exposure in a university setting. It is therefore somewhat 
counterintuitive that the findings for academic vocabulary development 
were mixed. 

One reason for this may be that even the least experienced writers in 
this study were highly proficient. By virtue of being deemed capable of 
doing postgraduate academic work through the medium of English, these 
students can be classed as advanced users of English, and this is additionally 
indicated by the fact that their texts were richly populated with academic 
vocabulary. As Hyltenstam & Abrahamsson (2012) note, research on very 
proficient L2 learners is in short supply compared with the voluminous 
body of second language acquisition research on learners at lower profi- 
ciency levels. However, it is reasonable to expect their learning to progress 
at a slower pace, simply because they have less ground to cover. In other 
words, there may be a phenomenon at play akin to a ceiling effect, accord- 
ing to which the year two texts did not show much greater lexical diversity 
and sophistication because the year one texts were already satisfactory in 
that regard. 

Similarly, it may be thought that these students had relatively limited 
opportunities for vocabulary development. The EMI environment provides 
a context in which only incidental language acquisition can occur, rather 
than an EAP/TEFL environment where language development is the target 
of explicit instruction. As a result, opportunities for language learning are 
closely linked to exposure to the linguistic features that are candidates for 
learning. Less proficient learners have more opportunities for exposure to 
new forms than advanced learners, precisely because more of what they are 
exposed to is new. In the case of the high-register academic vocabulary that 
was the focus of the present investigation, the opportunities for exposure 
to the infrequent words decrease logarithmically, not arithmetically, once 
the first bands of very frequent words have been learned. 


5 Conclusions 


This article has reported an investigation into the academic vocabulary 

knowledge of students in an EMI setting. Students’ knowledge of aca- 

demic vocabulary is important in this context because it is essential both 
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for adequate comprehension of academic texts and for producing register- 
appropriate assessment work. As a consequence of the fact that study in 
the EMI environment places demands on students’ receptive and productive 
academic vocabulary knowledge, it is an aspect of linguistic proficiency 
which could reasonably be expected to develop over the course of their 
studies. A measure of students’ productive academic vocabulary is therefore 
a useful indicator (though by no means the only one) of two important 
factors: students’ preparedness for academic study, and their development 
in English. 

To the extent that the findings presented in this paper speak to prepared- 
ness, they permit an optimistic interpretation: academic vocabulary items 
accounted for approximately 20% of all tokens, a rather higher figure 
than that found in many earlier studies. Although knowledge of academic 
vocabulary alone cannot be interpreted as evidence that students are equal 
to the challenges of study through the medium of English, a more cautious 
claim can be made: there is no reason to believe that this cohort of students 
lacks an adequate knowledge of academic vocabulary. 

The high level of coverage also provides support for the principles un- 
derlying the construction of the AVL. By including items which occur in 
academic texts more frequently than in general ones, and by excluding 
items which occur disproportionately frequently in some disciplines only, 
the AVL is designed to give a better representation of general academic 
vocabulary than earlier lists, and the incidence of AVL items in the present 
corpus provides indirect evidence that the AVL behaves the way it was 
intended. While this does not resolve all of the problematic aspects of the 
notion of an academic core vocabulary (cf. Hyland/Tse 2007), it suggests 
that, in circumstances where an academic vocabulary list is necessary or 
desirable, for pedagogical or research purposes, the AVL is the list of choice. 

Perhaps more significantly, this measure of productive academic vocabu- 
lary gives no support for the idea that international students and local Swed- 
ish students differ in their abilities in English. This is reassuring given the 
fact that the economic and policy imperatives in Swedish higher education 
(and reflected elsewhere in Europe) will for the foreseeable future lead to 
an increase in inward student mobility. 

With regard to vocabulary development between the first and the sec- 


ond year, evidence was limited; there were modest gains by some measures 
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and none by others. This is a finding of relevance given the current rapid 
expansion of EMI, and the twin motivations behind it. EMI is expected to 
be both a tool to facilitate mobility in higher education and a vehicle for 
improved English language skills on the part of participants but this study 
of academic vocabulary knowledge provides little indication that the latter 
ambition is realized, at least in the context under investigation. 


6 Pedagogical implications 


In this volume, with its focus on the pedagogical aspects of assessment, the 
pedagogical implications of the findings merit exploration. However, the 
EMI environment is complex in its pedagogical objectives. One objective 
of EMI is simply to enable the teaching and learning of subject matter by 
using English as an academic lingua franca. In many EMI contexts, though, 
an additional objective is to provide a context which facilitates students’ 
incidental acquisition of English. The pedagogical implications of students’ 
vocabulary knowledge and development are different for these two differ- 
ent objectives. 

In terms of content learning, these students appear to be well equipped 
with a productive knowledge of academic vocabulary sufficient to complete 
assessment tasks (and therefore by implication with a receptive vocabulary 
sufficient to read academic texts). This means that teachers (provided they 
have similar student profiles and communication genres) can concentrate 
on, for instance, promoting the critical reading of the disciplinary vocabu- 
lary. From a collaborative learning perspective, peer learning can enable the 
further exploration of the enhanced understanding of technical vocabulary. 

In the scenarios where the EMI context involves an element of col- 
laboration or contact between language lecturers and subject lecturers, 
the language lecturer might help the subject lecturer highlight the way in 
which academic vocabulary serves to carry the disciplinary argument. Such 
a shared focus would help students articulate the necessary disciplinary 
connections between argumentative components. A subject lecturer might 
contribute with useful insights for prompts, exercises, and classroom as- 
sessment techniques focused on exploring technical vocabulary. 

With respect to language development, teachers may conclude that basic 
academic vocabulary knowledge can be taken as confirmed. They can there- 
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fore use this apparent communicative resource of academic vocabulary as a 
stepping-stone to explore the remaining dimension of written disciplinary 
communication. For example, they might have students extract and master 
the technical vocabulary in the texts they encounter via basic critical reading 
using genre and corpus analyses. 

The productive knowledge of the frequent academic vocabulary items 
demonstrated here could also be a potential stepping-stone toward com- 
mand of the less frequent AVL items. However, the evidence of this study 
is that development along those lines does not happen automatically, and 
indeed there is no reason to suppose it should, given that opportunities for 
exposure to infrequent vocabulary are limited. A key pedagogical implica- 
tion of these findings is therefore that incidental acquisition is unlikely to be 
accidental, and that teachers who hope their students’ academic vocabulary 
will develop during an EMI course should create opportunities for exposure 
to and practice of a broader range of academic lexis. 

This study underscores a reality of many EMI settings. EMI is intended 
to be a de facto form of Content and Language Integrated Learning (CLIL) 
but while CLIL settings work actively both with content knowledge and 
with language development, in EMI the expectation is frequently that the 
preconditions for incidental language acquisition are put in place simply 
by dint of offering instruction in English. This study has provided evidence 
that those expectations are not entirely justified. 
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Assessing writing in vocational education 
and training schools: 
Results from an intervention study 


Abstract 

English: In this paper, we describe the assessment of writing in vocational educa- 
tion and training (VET) schools within the scope of an intervention study carried 
out in the German-speaking part of Switzerland. In literacy education within VET 
schools, the diversity of students’ linguistic backgrounds is not sufficiently taken 
into consideration. To address this lack, a teaching approach based on process- 
oriented writing instruction incorporating both German as L1 and German as L2 
approaches was developed and evaluated. The paper focuses on the instruments 
used for the assessment of students’ writing competence at three stages and describes 
the writing test as well as the scoring procedures applied. Reflections on the quality 
of the writing competence scale (Cronbach’s alpha: .82-.85) and some of the main 
results showing a significant increase in writing competence in the experimental 
group compared to the control group are presented alongside a discussion of the 
potential value of our assessment procedures. 


German: Dieser Beitrag hat die Erhebung von Schreibkompetenz im Rahmen einer 
Interventionsstudie an Berufsfachschulen der Deutschschweiz zum Thema. Im allge- 
meinbildenden Unterricht dieser Schulen findet sich ein hoher Anteil von Lernenden 
mit mehrsprachigem Hintergrund und unkonventionellen Sprachbiografien, was in 
der Förderung der Sprach- bzw. Schreibkompetenz jedoch nur wenig berücksichtigt 
wird. Aus diesem Grund wurde ein Konzept der prozessorientierten Schreibdidaktik 
entwickelt, das Mutter- und Zweitsprachenunterricht mit Schreib- und Sprachför- 
derung zusammenbringt und evaluiert. Der Artikel konzentriert sich auf eine Dar- 
stellung der Instrumente zur Erhebung und Bewertung der Schreibkompetenz, was 
eine Beschreibung der Schreibtests und des Scoring-Verfahrens beinhaltet, ebenso 
wie auf Überlegungen zur Qualität der Schreibkompetenz-Skala (Cronbachs Alpha: 
.82-.85). Die Hauptergebnisse, die unmittelbar nach der Intervention einen signifi- 
kanten Zuwachs der Schreibkompetenz der Experimentalgruppe im Vergleich zur 
Kontrollgruppe zeigen, werden vorgestellt, bevor der Artikel mit einer Diskussion 
der Ergebnisse schließt. 
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1 Introduction 


Literacy development in vocational education and training (VET) schools 
is a field that has only recently attracted research (Becker-Mrotzek/Kusch/ 
Wehnert 2006; Efing 2008) in spite of the high value that is placed on the 
dual education system in German-speaking countries. In Switzerland, an- 
nually more than 65% of all students pass through one of the 2- to 4-year 
VET programmes, in which they spend between one and two days! a week 
in school and the remainder at work (SBFI 2015: 4 ff.). In today’s digital and 
knowledge-based society, reading and writing skills become increasingly 
important in VET and in professional life (Miiller 2003; Jakobs 2008). In 
2010, the Swiss Business Federation published the results of a survey of 
771 Swiss enterprises of various sizes and representing numerous business 
sectors. Participating businesses highlighted that apprentices had a general 
lack of mathematical and especially language skills at the beginning of their 
training. This concerned not only apprentices with a migration background, 
but also native VET students (economiesuisse 2010: 3 ff.). 

For the majority of VET students, reading and writing are complex 
cognitive and linguistic challenges that they do not always master success- 
fully (Nodari 2002: 11). Although studies focusing on VET schools are 
not available, we must assume that a sizeable number of VET students, 
especially those in apprenticeships that are less demanding regarding school 
achievement, come close to what, in Shaughnessy’s (1977) understanding, 
are “basic writers”. They are little acquainted with the formal require- 
ments and conventions of Swiss standard German and unaccustomed to 
expressing their ideas in written form. They are not overly motivated to 
cope with language-related tasks and have often been unsuccessful writers 
in their previous educational experience. As scholastic achievement is one 
of the main selection criteria for the apprenticeships that students choose, 
we must also assume that those students gaining access to apprenticeships 
that are more demanding with respect to language skills usually have a 
higher proficiency in German. 


1 VET can also be completed at a full-time vocational school. In the French- 
speaking and Italian-speaking parts of Switzerland, the proportion of full-time 
vocational schools is higher than in German-speaking Switzerland. For further 
information about VET in Switzerland, see Swisseducation (2015). 
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It is more than motivation and the kind of apprenticeship, however, 
that explains variation in language skills. Many VET students come from 
families with a modest level of education and/or a migration background. 
Adolescents with an immigrant background have usually grown up in mul- 
tilingual environments using several languages simultaneously and learned 
German as a second or third language. Furthermore, an additional chal- 
lenge is that both native and migrant-background students must cope with 
diglossia in Switzerland. There is a large gap between the written language 
and the local dialects spoken in Switzerland with their markedly different 
lexis, grammar and idioms as compared to Swiss standard German (Am- 
mon/Bickel/Ebner 2004; Dürscheid/Giger 2010; Diirscheid/Elspass/Ziegler 
2011). Today, local Swiss dialects are also preferred in informal written 
communication such as SMS and e-mail. As local Swiss dialects are not 
standardized with respect to grammar and spelling, young people are not 
accustomed to regulated language usage and are not overly motivated to 
comply with the norms of Swiss standard German (Sieber 2013). In spite 
of the linguistic diversity in VET schools in German-speaking Switzerland, 
the teaching of Germani/Jliteracy still follows approaches designed for mono- 
lingual groups of native speakers, and teachers are trained to teach classes 
consisting of purely mother-tongue students (Belke 2001: 2) without ac- 
counting for their pervasive multilingualism. 

At Swiss VET schools, the teaching of language and communication is 
integrated into a multidisciplinary subject called ABU “allgemeinbildender 
Unterricht”, which can be translated as “general education” in contrast to 
vocational education subjects (BBT 2006). ABU allows teachers to focus on 
a wide range of subjects of general interest such as economics (e.g., deal- 
ing with money), civics (e.g., voting), or law (e.g., signing a contract). The 
teaching of German/literacy is integrated into these fields of content. Even 
though school curricula include instructions on the relationship between 
language/literacy teaching and the teaching of general education content, 
teachers are quite free to create their own teaching units into which they 
can integrate language instruction and the teaching of writing. 

The assessment procedures reported on in this paper are part of an inter- 
vention study in which a new way of teaching writing in VET schools was 
introduced. The basic philosophy behind the teaching method we suggested 


was a process approach largely following the instructional model outlined 
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in American secondary and higher education (cf. Pritchard/Honeycutt 2006) 
in the 1970s, and which was later adopted in other countries (cf. Ruhmann/ 
Kruse 2014). This approach was integrated with an L2 approach to writing 
to cater to the lack of essential linguistic skills in Swiss standard German, 
which holds true for the majority of students (both native and non-native). 

To describe which decisions led to the construction of our measure- 
ment techniques, we will briefly review the literacy education situation in 
Swiss vocational schools and describe the nature of the intervention we 
conducted. This is followed by a description of the writing test and the 
scoring procedures. We then introduce the writing competence scale and 
detail some of the major results of the intervention. Finally, we show that 
the changes in the teaching of writing resulting from our intervention are 
well reflected in the writing competence scale, thus demonstrating it to be 
a valid instrument for the assessment of writing skills. 


2 Process-oriented writing between L1 and L2 in vocational 
schools 


In our project, we were looking for ways to introduce a process approach 
to writing in VET schools and add approaches from the teaching of German 
as an L2 in order to more effectively respond to the multilingual realities 
of the student population. 

The process approach revolutionized the teaching of writing in educa- 
tional contexts in the United States after Emig’s (1971) study The Compos- 
ing Process of Twelfth Graders. It offered a new perspective on writing by 
drawing attention away from the idealized prescriptions on what good texts 
are and directing it to the thinking process of the writer instead. Writing 
has since been seen as a way of generating text through several interrelated 
activities which are not processed in a linear way, as in earlier models of 
the writing process, but recursively. This means that writers tend to rethink 
their ideas repeatedly and proceed in several loops before they decide on the 
final form of the text. This paradigm shift was the starting point for research 
looking at what really happens during writing and is still seen as one of the 
cornerstones of writing pedagogy (Anson 2014; Ruhmann/Kruse 2014). 

From the 1980s on, the process approach was accompanied by cognitive 
models of writing, initiated by Hayes & Flower (1980), Flower & Hayes 

Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 


Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Assessing writing in vocational education and training schools 77 


(1980; 1981), Bereiter (1980), Bereiter & Scardamalia (1987), and others, 
that view writing as a series of cognitive activities such as idea generation, 
structuring ideas, planning, proposing ideas, and translating ideas into 
word strings. This approach placed a strong emphasis on problem-solving 
activities and writing strategies. Recently, several new elements have been 
absorbed into the process approach, the most important being peer feed- 
back, collaborative writing and reflective techniques (Pritchard/Honeycutt 
2006; Ruhmann/Kruse 2014). 

Unfortunately, the shift from product to process has resulted in an in- 
creased reluctance towards the teaching of language in the writing class. In 
cognitive process models (in the tradition of Hayes/Flower 1980), linguis- 
tic activity is viewed as a means of translating thought into text but not 
as a part of thinking, idea generation and meaning making. All essential 
processes are thought to be cognitive or computational in nature, so that 
there is no particular need to pay attention to the teaching of language for 
writers. As a result, language support has almost disappeared from the 
teaching of writing. If language support is given to writers, it is likely to 
happen in courses offered specifically for second-language writers who are 
seen as needing language skills (see, for instance, the work of Swales 1990; 
2004; Hyland 2000; 2011). 

Recently, it has become clear that language instruction has to find its way 
back into the teaching of writing in L1 classes (Steinhoff 2007; Pohl 2007; 
Locke 2010; Myhill 2010; 2012; Feilke 2012; 2014; Anson 2014). Hyland 
(2011: 24) expresses this need most emphatically when he claims that 

“teachers of writing clearly need to be teachers of language, as it is an ability to 

exercise linguistic choices in the ways they treat and organize their topics for par- 


ticular readers [...]. A knowledge of grammar, focusing on how students can modify 
meanings in distinct and recognizable ways becomes central to learning to write.” 


Hyland uses the term grammar in a broad sense here, referring to the lin- 
guistic means necessary for text production. In the case of VET schools, his 
suggestion seems particularly relevant, as students with multilingual back- 
grounds may lack the essential linguistic means for their task of learning, for 
example, business communication. In our approach, we drew on teaching 
units from German as an L2 to support the development of students’ lan- 
guage skills, and provided the students with a variety of focused linguistic 


elements that they could use when formulating or revising text. Materials 
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(e.g., reading assignments) to prepare writing tasks were also handed out 
to the students to instruct them on the writing processes/routines, which 
also include linguistic forms and rhetorical elements. We followed the idea 
of Feilke & Lehnen (2012) that learning to write means building routines 
when using rhetorical and structural elements for expressing ideas rather 
than simply ‘running’ cognitive functions as Hayes & Flower (1980) sug- 
gest. For this approach, it is important to focus on key linguistic elements 
that are needed to solve a task, and allow students to develop routines 
during writing and revising. 
The approach we developed was based on the following guidelines: 


e As a basis, we chose a procedure from the process tradition (Anson 
2014; Ruhmann/Kruse 2014) and deconstructed the writing process into 
distinct actions that can be connected individually in the development of 
a unique strategy. 

e We placed a strong emphasis on social activities and collaborative learn- 
ing to make writing a rewarding group enterprise. 

e We took care to create authentic prompts which connected writing to 
real-life situations of the writers. 

e From second-language learning, we added the principle of preparation 
(“Vorentlastung”) in which reading activities were presented before the 
actual writing task. 

e We followed a scaffolding strategy (Gibbons 2002) by accompanying the 
writing process with support for the linguistic tasks that had to be solved. 

e We connected oral and written activities to create synergies between both 
modes of text production and reception. 


We developed this teaching strategy together with the regular teachers, 
thereby allowing them to integrate the new curriculum into the require- 
ments of their existing teaching plans and their individual teaching styles. 
The teachers were introduced to the basic philosophy of our research ap- 
proach and then co-constructed the new teaching units for all stages of the 
curriculum. Altogether, we developed three major teaching units which 
covered 16 teaching hours (the equivalent of 33% of the overall time avail- 
able for ABU teaching in a semester). The teachers’ experience with and 
knowledge of the target group were very important for the implementation 
of our concept into teaching practice. 
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The three basic units were constructed as open scenarios in which writ- 
ing fulfilled a core function, but was seen as a means of reaching the goals 
of an overarching activity. There were three such scenarios that shared a 
common general structure. 

In the first scenario, students were asked to produce portraits of each 
other - to be printed in a brochure along with pictures — that would be 
presented to their families and at their workplaces. In the second scenario, 
students were asked to write a letter of complaint responding to an exces- 
sive bill from their mobile phone provider. In the third scenario, students 
were requested to write a letter to a real company that they would actually 
visit on a class excursion. These letters were written individually and then 
the best from each class was selected to be sent to the company. The best 
letter from all the classes was awarded a prize of 500 Swiss francs, which 
could be spent during the excursion. 

Each of these tasks required many classroom hours to prepare, carry out 
and evaluate. The main recurring elements were: 


e presenting the task 

e discussing the problems involved 

e providing materials to solve the problems (techniques, preparatory texts, 
search operations, etc.) 

e integrating ‘focus on language’ exercises into the writing process which 
raise awareness of language skills and their importance for the subse- 
quent writing or prewriting phase 

e initiating reflection on text procedures that take place in writing activities 

e generating and structuring ideas, draft writing 

e peer feedback with subsequent revision 

e writing the final version 


There was frequent alternation between individual and group work, and 
each paper went through several cycles of consideration before students 
settled on the final version. In this way, participants were led through stages 
of generating ideas, planning, discussing and revising several times.” 


2 For further information on the intervention and the teaching materials used, see 
Hoefele/Konstantinidou (in prep.). 
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3 Study design and results 


The intervention study took place in 18 different classes in three VET 
schools in Switzerland (N = 318). Nine of these classes received regular 
ABU teaching (control group) while nine classes received a special treatment 
of process-based teaching of writing (experimental group). Teaching in the 
control group followed the usual principles of general education classes in 
VET schools connected with several writing assignments which were graded 
and aimed at supporting writing development. The general study design 
can be described as a pre-test/post-test/control group design with follow-up. 
Experimental and control groups were compared at the beginning of the 
intervention, at the end of the intervention, and after a follow-up period of 
3 months. The test procedures and assessments were not part of the teaching 
assignments, but were carried out on separate occasions and in an identical 
manner for both groups. Testing took place in August 2013 (pre-test), in 
February 2014 (post-test), and in June 2014 (follow-up test). 

Because of school circumstances (practical and administrative issues), 
classes (natural groups) rather than students were randomly assigned to the 
experimental and control group respectively (quasi-experimental study). In 
order to avoid selection effects or major differences between the interven- 
tion and the comparison group that might explain differences (or lack of 
differences) in the post-test and follow-up comparisons, the project team 
created a control group that matched the experimental group with respect 
to the apprenticeships included in the sampling. In this manner, six differ- 
ent apprenticeships (hairdressers, information/documentation assistants, 
lab assistants, logisticians, multi-skilled mechanics and painters) from one 
group of classes were assigned to the experimental group, and those from 
their parallel classes at the same school were assigned to the control group.’ 
Furthermore, variables that might confound the results, such as gender, im- 
migration and linguistic background, were taken into consideration (Bortz/ 
Döring 2006: 524 ff.). 


3 In the school year of the intervention, there was no parallel class for the in- 
formation/documentation assistants. For this reason, a vocational class of lab 
assistants was assigned as the control group. According to school coordinators, 
these two vocational classes are comparable regarding school achievement and 
motivation. 
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As mentioned above, the participating teachers of the experimental group 
were introduced to process-oriented teaching of writing in both L1 and 
L2 situations and prepared to participate in the intervention study (as me- 
diators). In cooperation with them, a new writing curriculum including 
three extended writing tasks based on the scenarios mentioned above was 
developed and tested against a traditional curriculum. The main research 
question concerned differences between the students in the experimental 
group and the control group regarding their writing competence imme- 
diately after and four months after the intervention. It was assumed that 
immediately following the intervention (and four months later) students 
in the experimental group would show better writing skills than students 
in the control group and that they would organize, plan and revise their 
writing better. Students in the experimental group were also expected to 
have more positive attitudes towards writing and be more self-confident, 
self-regulated and motivated writers.* 


3.1 The writing test 


There are several possible reference points for assessing writing skills at 
VET schools (Weigle 2002). A basic requirement of educational research 
is to arrive at methodologically sound estimations of students’ abilities 
at all levels of educational systems. Very little, however, is known about 
the level of literacy skills to be expected from students in VET schools, in 
which students from very heterogeneous professional fields (ranging from 
multi-skilled mechanics and information/documentation professionals to 
hairdressers and logisticians) are educated and where the teaching of writ- 
ing is not as clearly defined as in primary education. 

A more reliable reference point for the assessment of writing is the teach- 
ing situation and the literacy curriculum running in the background, no 
matter how ill-defined it may be. Connecting assessment with teaching 
may also lead to better instruction. In turn, better instruction can provide 
information regarding the validity of scales with respect to their sensitivity 
to the kind of change our intervention leads to. In order to estimate the 


4 Results regarding attitudes and other psychological aspects are available but are 
not the focus of this article. 
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impact of the new teaching strategy, we developed assessment tools that 
reflect both the particular level of the vocational students and the nature 
of the intervention. 

The writing test was developed in three stages: design, operationalization 
and administration (Bachmann/Palmer 1996, quoted in Weigle 2002: 77). 
During the design stage, the general curriculum (BBT 2006) and the cur- 
ricula of the participating schools were studied in order to determine what 
kind of texts students are required to write in the first year of VET. Further- 
more, a focus-group discussion with VET students allowed us to identify 
their writing needs in everyday environments outside school. We expected 
this procedure to lead to authentic writing prompts connected to the stu- 
dents’ real-world concerns. We selected a persuasive writing/argumentation 
task (Becker-Mrotzek/Bottcher 2012: 218 ff.) which required literacy skills 
that came closest to the kind we intended to measure. Students were asked 
to compose formal letters (motivational letters) in which they had to argue 
for a cause and convince the intended audience to support it. 

The writing test consisted of three different authentic prompts that were 
specified in cooperation with the teachers in the operationalization stage. 
The selection of authentic prompts was intended to arouse the interest of 
the test-takers (interactiveness). The prompts were as follows: (1) at the 
pre-test stage, students were to write a letter to the school management 
in order to win a school contest for a four-day stay for the whole class in 
the French-speaking part of Switzerland as a prize; (2) in the post-test, the 
recipient of the motivational letter was the education office, which finances 
vocational students’ sojourns in Ticino; and (3) in the follow-up test, the 
letter had to be directed to the students’ workplace mentor who had to be 
persuaded to grant a paid leave for the participation in a school sojourn 
in the Italian-speaking part of Switzerland. Formal letters reflect the multi- 
dimensional nature of writing and are conventionalized enough to allow 
for the development of clear rating criteria which contribute to the validity 
of the instrument (see section below). 

The writing test was presented as a booklet of eight pages including pre- 
writing pages (planning and draft pages) as well as the criteria for the text 
evaluation. As is common in test situations at VET schools, all assessments 
were written by hand directly in the booklet provided. For the integration 


of process-oriented writing elements into the assessment, as in the National 
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Assessment for Education Progress (NAEB 2007), students were given a 
substantial amount of time for completing the task and were invited to 
undertake planning and revising activities before starting to write (e.g., On 
this page you can make notes and plan your writing.). The writing time of 
each student was recorded.° 

To prepare the main survey, the writing tasks were tested in a pilot 
study (administration stage) with 118 VET students from the same ap- 
prenticeships as those participating in the main study. The results of the 
pilot study (empirical item investigation, factor and reliability analysis, 
inter-rater reliability) led to changes and adaptations in the wording of the 
writing tasks and in the scoring criteria.° Additionally, in order to confirm 
the validity of the measurement instrument, we asked the VET teachers to 
verify whether pilot study test scores (see Section 3.2) corresponded to the 
students’ general achievement in writing (Weigle 2002: 49). This result was 
very encouraging since teachers confirmed that test scores corresponded to 
the writing achievement for more than 80% of the students. 

The writing test was administered by trained test administrators. A test 
administrator manual based on previous international studies (PISA, PIRLS, 
ICCS) described their roles and responsibilities and included instructions 
on the distribution of the student testing instruments according to student 
tracking forms, the supervision of the testing sessions and the recording of 
students’ participation (Schulz/Ainley/Fraillon 2011: 91). 


3.2 Scoring procedures 


The development of scoring criteria also formed part of the operationaliza- 
tion stage of the writing test’s development. For the scoring procedure, a 


code book was developed. It was based on previous German studies, includ- 
ing DESI (Harsch et al. 2007) and VERA (code book available online at IQB 


5 The recording of writing times, as well as the assessment of planning and revising 
activities, were intended as an attempt to assess some writing process elements. 
These elements, however, were not taken into consideration when assessing the 
quality of the final written products. 

6 Adaptions of the scoring criteria mostly concerned the Correctness and Style 
subscales. 
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n.y.), the work of Becker-Mrotzek & Bottcher (2012) and Nussbaumer & 
Sieber’s (1994) Zurich Analysis Grid. 

As the main scoring criteria, six rating scales (or subscales) were chosen, 
which were assigned to three main dimensions: 


e Linguistic Competence: subscales Correctness and Style 

e Genre Competence: subscales Formal Conventions and Structure/Cohe- 
rence 

e Pragmatic Competence: subscales Content and Communicative Impact 


A limited number of scoring criteria (here: six) is suggested by Becker- 
Mrotzek & Böttcher (2012: 128) and Baurmann (2008: 133) because they 
can easily be adapted to writing tasks and the objectives of evaluation, which 
increases the practicality of the scoring procedures (Weigle 2002: 138). 

Each text was rated as a whole using each of the six subscales. For the 
possible scores on each subscale, a range from 0 to 4 points was chosen with 
4 points indicating complete mastery and 0, no mastery. Very short texts 
(under 50 words) were also rated with 0 (see Tables 1, 2 and 3). Mastery 
levels were defined a priori and are described in the tables. This allowed us 
to make inferences about test takers’ writing competence on an absolute 
scale level and not only in relation to other test takers (Weigle 2002: 125). 
The Writing Competence score is the sum of the six subscale scores and 
amounts to a maximum of 24 points. Raters used the full range of scores 
for all six rating scales except for the Formal Conventions scale, where none 
of the students was given zero points. 

The overall scoring strategy may be called ‘analytic’ as we decomposed 
the construct of Writing Competence into several distinct rating scales (Wei- 
gle 2002: 109 ff.). Each of the six rating scales themselves, however, may be 
characterized as a combination of analytic and holistic scoring, as some of 
them demand a holistic judgment while others are based on distinct criteria 
or the number of errors. While, for example, the Correctness and Formal 
Conventions rating scales are quite analytic and include error counting, 
the Communicative Impact rating scale is holistic, asking the rater for a 
judgment on the extent to which the text would convince the recipient. 
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Table 1: ‘Linguistic Competence’ scoring criteria and descriptors (based on the 
DESI and VERA code books) 


Linguistic Competence 


Scoring 
Criteria 


Level 


Descriptor 


Correctness 


Texts at this level contain almost no spelling, punctuation or gram- 
matical inaccuracies (fewer than 2). Syntax is correct; sentences are 
complete. Declination, conjugation or tense errors do not occur. 


Texts at this level contain few spelling, punctuation or grammatical 
inaccuracies (fewer than 5) that do not disturb reading flow or com- 
prehension. Syntax is rarely incorrect. 


Texts at this level contain multiple spelling, punctuation or gram- 
matical inaccuracies (more than 5) OR: Texts at this level have few 
spelling, punctuation or grammatical inaccuracies (fewer than 5) but 
these inaccuracies disturb reading flow or comprehension. Occasio- 
nally, syntax errors occur. 


Texts at this level contain multiple spelling, punctuation or gramma- 
tical inaccuracies that disturb reading flow or comprehension. Syntax 
is often incorrect. 


Texts are too short (fewer than 50 words) and cannot be scored. 


Style 


Texts at this level contain a vocabulary range that is both broad and 
adequate for the task and genre. Word repetitions hardly ever occur. 
Word choice is mostly accurate, appropriate and without dialectal 

influences. Formulations are objective. Register is formal and polite. 


Texts at this level contain a relatively broad vocabulary range that is 
mostly adequate for the task and genre. Word repetitions rarely occur. 
Word choice is accurate, appropriate and without dialectal influences. 
However, communication patterns characteristic of spoken language 
occasionally occur in the text. Formulations are mostly objective. 
Register as a whole is formal and polite. 


Texts at this level contain a restricted vocabulary range. OR: Texts at 
this level contain vocabulary that is inadequate for the task and genre. 
Occasionally, word repetitions occur. Word choice is on the whole 
accurate. However, texts contain errors that negatively affect com- 
prehension and/or communication patterns characteristic of spoken 
language often occur. The text is at times narrative, and formulations 
are not objective enough. Register as a whole is formal and polite. 


Texts at this level contain a very restricted vocabulary range. OR: 
Vocabulary is mainly copied from the writing task. AND/OR: Word 
repetitions occur frequently. AND/OR Vocabulary is inadequate for 
the task and genre. Word choice is clumsy and colloquial. The text 
is mostly narrative, and formulations are not objective. Register as a 
whole is inappropriate and too informal. 


Texts are too short (fewer than 50 words) and cannot be scored. 
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Table 2: ‘Genre Competence’ scoring criteria and descriptors (based on the DESI 
and VERA code books) 


Genre Competence 


Scoring 
Criteria 


Level 


Descriptor 


Formal Conventions 


Texts at this level fully conform to the formal conventions of the text 
genre (letter). The return and recipient addresses are correct (deliver- 
able) and in the correct place. Place and date are present and in the 
correct place. The form of address and salutation are appropriate 
and both are in the correct place. The letter is signed. 


Texts at this level mostly conform to the formal conventions of the 
text genre (letter). Only one of the above-mentioned conventional 
elements is missing, is incorrect/inadequate for the recipient or is not 
in the correct place. 


Texts at this level partially conform to the formal conventions of 

the text genre (letter). Two of the above-mentioned conventional 
elements are missing, are incorrect/inadequate for the recipient or are 
not in the correct place. 


Texts at this level do not conform to the formal conventions of the 
text genre (letter). More than two of the above-mentioned conven- 
tional elements are missing, are incorrect/inadequate for the recipient 
or are not in the correct place. 


Texts at this level do not contain any of the correct conventional ele- 
ments of the text genre (letter). 


Structure/Coherence 


Texts at this level are meaningfully and logically structured and can 
be read fluently. Structural elements are easily identifiable and in a 
logical order. Sentence structure is varied and sufficiently sophisti- 
cated for the task. Texts are completely coherent with diverse and 
appropriate connectors. 


Texts at this level are meaningfully and logically structured and can 
mostly be read fluently. Structural elements are identifiable and in a 
logical order. Sentence structure is mainly simple, though there may 
be combinations of main and subordinate clauses. Various connec- 
tors are used appropriately. 


Texts at this level are mostly meaningfully and logically structured, 
but show a lack of rhetorical fluency at times. Structural elements are 
identifiable, though not in a logical order. Generally, main clauses are 
present but subordinate clauses occur only infrequently. OR: Subor- 
dinate clauses are overly complicated. Some connectors are incorrect. 


Texts at this level have little or no logical structure. Structural elements 
are barely identifiable and not in a logical order. The texts consist 

only of main clauses connected with and or of main and subordinate 
clauses that are either not connected or incorrectly connected. 


Texts are too short (fewer than 50 words) and cannot be scored. 
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Table 3: ‘Pragmatic Competence’ scoring criteria and descriptors (based on the 
DESI and VERA code books) 


Scoring 
Criteria 


Level 


Pragmatic Competence 


Descriptor 


Content’ 


Texts at this level refer to all issues in sufficient detail (e.g., 
reason for writing, class description, description and justification 
of intentions, gratitude). The purpose of writing is clear. 
Argumentation is strong and weighty. 


Texts at this level refer to all issues (e.g., reason for writing, class 
description, description and justification of intentions, gratitude) 
and deal in sufficient detail with at least three of them. The purpose 
of writing is clear. Argumentation is good and weighty enough. 


In texts at this level, only one issue is missing. However, all other 
issues are dealt with in sufficient detail. OR: Texts at this level refer 
to all issues (e.g., reason for writing, class description, description 
and justification of intentions, gratitude), but two of them are not 
dealt with in sufficient detail. 


In texts at this level, more than two issues are missing. However, all 
other issues are dealt with in sufficient detail. The purpose of writing 
is clear. OR: Texts at this level refer to all issues (e.g., reason for 
writing, class description, description and justification of intentions, 
gratitude) but not in sufficient detail. 


Texts cannot be scored. 


Communicative Impact 


Texts at this level would definitely persuade the reader. They are 
very convincing regarding content, form and rhetoric. 


Texts at this level are likely to persuade the reader. They are 
convincing regarding content, form and rhetoric. 


Texts at this level could fail to persuade the reader. They are only 
partly convincing regarding content, form and rhetoric. 


Texts at this level are likely to fail to persuade the reader. They are 
only slightly convincing regarding content, form and rhetoric. 


Texts cannot be scored. 


7 The evaluation of the content is task-dependent. The description in the table is 
rather general compared with the description in the project code book. 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


88 L. Konstantinidou / J. Hoefele / O. Kruse 


To ensure reliability, all tests were scored independently by two raters. 
Test scoring took place after the study was completed. The three raters 
participating in the scoring procedures received intensive training in which 
each scoring criterion was discussed in detail and more than 30 randomly 
selected texts were scored jointly. The aim of the training was to achieve 
high agreement between the raters regarding their understanding of the 
scoring criteria. After training, texts were assigned to the three raters by 
VET class. A main rater was appointed for every class, but test versions 
(pre-test, post-test, follow-up test) and experimental/control conditions 
were permuted and assigned randomly to avoid rater bias. 

While independent scoring was used in the pilot study,? we preferred the 
method of consensus scoring for the main survey. In accordance with this 
method, raters score independently in a first step but subsequently have the 
opportunity to discuss their results on every criterion in rater teams (main 
rater A and rater B) in order to arrive at a consensus (Robinson 2000; NAEP 
2008). In our study, in case of disagreement, the results of the main rater 
were counted in the project data set. In any case, situations where raters did 
not agree were rare (see the results of inter-rater reliability below). 

The degree of agreement between the two raters provides a measure of 
the reliability of the scoring process (Schulz/Ainley/Fraillon 2011: 96). To 
measure inter-rater reliability, Cronbach’s alpha coefficient was calculated, 
which provides an estimate of the consistency of ratings across multiple 
judges (Stemler 2004). Because of the consensus scoring, a high inter-rater 
reliability was expected. This expectation was confirmed by the results, 
where Cronbach’s alpha ranged from .98 to 1.0 for all subscales, and the 
overall Writing Competence scale showed an almost optimal inter-rater 
reliability (Cronbach’s Alpha: .997). 

Examples of low, medium and high writing competence of VET students 
are presented below. The three texts were produced by students from the 
same VET class and were collected in the post-test phase. 


8  Inter-rater-reliability results (Cronbach’s alpha .93) from the pilot study showed 
a high agreement between the raters regarding the Writing Competence scale. 
Only for the subscales Correctness and Style, Cronbach’s alpha was under .70. 
This led to adaptations and clarifications of the subscales for the main survey. 
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Vorname Name 
21 Wolfstreet 
3013 New York 


Generalsekretariat der Bildungsdirektion 
Wachenplatz 2 
8090 Zürich 


New York, 30,1,14 

Betriebsbesichtigung. 

Guten Tag wir sind die Musterklasse, wir bestehen aus 10Jung die Zurzeit den Musterberuf lernt wir 
sind im 1. Lehrjahr und haben schon viel erfahrung. Wiso wir teilnehmen möchten, ganz einfach wir 
sind eine Disziplienirte und Lernfähige klasse die Sehr gerne mehr erfahrung sammel möchte. 


Wir würden uns sehr freuen über ein positive antwort. 


Liebe grüsse ... 
(Unterschrift) 


Vorname... 

Musterstrasse 11 

0000 Musterstadt 

Generalsekretariat der Bildungsdirektion 

Wachenplatz 2 

8090 Zürich 

30.1.2014 

Betriebs und Berufsschulbesichtigung 

Sehr geehrte Damen & Herren 

Wir sind eine Musterklasse mit 11 Schülern aus Musterstadt im Kanton Zürich. 

Wir sind eine Motivierte Klasse die zusammenhält und zielorientiert denkt, gerne neue Berufe oder 
Orte kennerlenen, sich darüber Infos sammelt und gedanken machen. Da das Tessin ein anderer 
Kanton, eine andere Sprache hat, nimmt es uns Wunder wie man dort den Berufsunterricht gestalltet 


und wie sie Arbeiten. 


Was die Arbeit bzw. die Berufe sind wir offen für Neues. Wir sind auch gespannt was Firmen dort 
unten im Süden alles herstellen und wo sie auf der Welt tätig sind. 


Wir würden uns uber eine Positive Antwort riesig Freuen} 


Freundliche Grüsse 
(Namen aller Mitschüler und Musterklasse) 
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Generalsekretariat der Bildungsdirektion 
Wachenplatz 2 
8090, Zürich 


Musterklasse, Musterberufsschule 
Musterstrasse 1 
1234, Ziirich 


Musterstadt, 30.1.14 
Antrag Berufsschulaustausch 
Sehr geehrte Damen und Herren 


Im Namen der Musterklasse der Musterberufsschule schreibe ich Ihnen, um Gelder fiir einen 
Berufsschulaustausch mit dem Tessin zu beantragen. 


Wir sind eine Klasse bestehend aus interessierten ...-Auszubildenden des ersten Lehrjahres. Als wir 
davon erfahren haben, dass die Möglichkeit besteht als Klasse an einem Berufsschulaustausch 
teilzunehmen waren wir sofort davon begeistert. Unser Plan sieht vor, dass wir mit dem von Ihnen 
erhaltenen Geldern eine Berufsschule im Tessin und mehrere umliegende Betriebe besuchen. Auf 
diese Weise ist es uns möglich den Klassenzusammenhalt zu stärken, unser Wissen über die 
Kulturlandschaft der Schweiz zu festigen und noch zu erfahren, wie anderswo in Schulen und 
Betrieben gearbeitet wird. 


Wir sind Ihnen äusserst dankbar, dass Sie uns die Möglichkeit geben möchten, dies zu erleben und zu 
erfahren. 


Wir hoffen auf eine positive Resonanz. 


Freundliche Grüsse 
(Unterschrift) 

In Verehrung der Musterklasse 
(Vorname und Name) 


Fig. 1: Examples of low, medium and high writing competence of VET students 


3.3 The writing competence scale 


As mentioned above, the writing competence scale was defined as the sum 
of the six rating scales and thus ranging from 0 to 24 points. To check the 
one-dimensionality of the scale, an exploratory factor analysis (principal com- 
ponent method with varimax rotation) was conducted for each of the test 
administrations (t0, t1 and t2) with the six rating scales as single items. The 
suitability of the data for this analysis was confirmed by the results of the KMO 
(Kaiser-Meyer-Olkin Measure of Sampling Adequacy: approx. .82 for all three 
administrations) and Barlett’s tests (.000). Based on Kaiser’s rule, which drops 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Assessing writing in vocational education and training schools 91 


all factors with eigenvalues under 1.0 (Bühner 2006: 200), one solid factor 
capable of explaining 57 to 60% of all the variables’ variances across the three 
administrations was selected. The consistency of the constructed scale was con- 
firmed by a reliability analysis, in which subjects with missing values on one of 
the items/rating subscales were excluded. The result of this analysis (Cronbach’s 
Alpha .82-.85 for the three administrations) indicates a highly reliable scale. 

Tables 4 to 6 show the main results of the factor and reliability analyses. 
Five of the six items/rating subscales have very good psychometric proper- 
ties and score high on the main factor, which supports the interpretation 
of the sum of scores as the Writing Competence scale. The subscales Com- 
municative Impact and Structure/Coherence are the most representative of 
the overall assessment score. 


Table 4: Writing Competence scale t0 (pre-test) 


Writing Competence scale t0 Exploratory Factor Analysis 
nz) Factor Corrected | Cronbach’s Alpha, 
loadings item-total if item deleted 

(Varimax) correlation 

Correctness 72 56 29 

Style .83 .69 .77 

Formal Conventions .36 .26 .87 

Structure/Coherence .86 Z5 75 

Content 77: .61 .78 

Communicative Impact .89 75) 7S 

Explained Variance 57% 

Cronbach’s Alpha 82 


Both the low correlation between the Formal Conventions score and the 
Writing Competence score (corrected item-total correlation: .26-.35) and 
the rise in Cronbach’s Alpha if the Formal Conventions subscale is deleted 
from the analysis (.87-.88) indicate that the Formal Conventions score 
does not discriminate enough between writers with high and low writing 
competence (Bortz/Döring 2006: 219). In other words, it is possible that 
participants who master the formal conventions of the genre receive low 
scores on the Writing Competence scale and vice versa. Similar results were 
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obtained in the pilot study. In spite of this, we decided to keep the item 
because formal conventions of letters belong to the curriculum of general 
education classes in VET schools and students are intensively trained to 
comply with them. In addition, formal conventions such as genre knowl- 
edge are important elements of writing competence according to theoretical 
writing models (e.g., Swales 1990; Russell 1997; Devitt 2004). 


Table 5: Writing Competence scale t1 (post-test) 


Writing Competence Scale t1 Exploratory Factor Analysis 

N) Factor Corrected | Cronbach’s Alpha, 
loadings item-total if item deleted 
(Varimax) | correlation 

Correctness .73 59 82 

Style 82 70 79 

Formal Conventions 47 35 87 

Structure/Cohesion .87 .75 .78 

Content .77 .62 .81 

Communicative Impact .86 .76 .78 

Explained Variance 58% 

Cronbach’s Alpha .84 


Table 6: Writing Competence scale t2 (follow-up test) 


Writing Competence Scale t2 Exploratory Factor Analysis 

(N= 236) Factor Corrected | Cronbach’s Alpha, 
loadings item-total if item deleted 
(Varimax) correlation 

Correctness .74 9 83 

Style 86 TS .80 

Formal Conventions 43 33 .88 

Structure/Cohesion .87 .76 .80 

Content .76 .64 .82 

Communicative Impact 89 .80 79 

Explained Variance 60% 

Cronbach’s Alpha 85 
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To test the quality of the scales, we carried out factor analyses for the 
whole sample for the three test administrations. Table 7 shows the re- 
sults for the Writing Competence scale for both groups (experimental and 
control) combined. In the pre-test, VET students had an average score of 
12.09 (SD: 3.7). Their performance was slightly better in the post-test and 
follow-up test while score variation was almost equally high for all three 
points of time (SD: 3.7-3.9). In the pre-test condition, VET-students’ scores 
ranged from 1 to 24; in the post- and follow-up tests, none of the students 
achieved fewer than 3 points. In the follow-up test, no students achieved 
the maximum score. Skewness and kurtosis give an impression of the scale’s 
distribution. While in the pre-test, the Writing Competence scale is slightly 
heavy-tailed, in the post- and follow-up tests, its distribution barely differs 
from the norm (see Table 7). 


Table 7: Writing Competence Scale - descriptive statistics for experimental and 
control groups combined 


N | Min | Max M SD Skewness Kurtosis 


Stat. | Stat. | Stat. | Stat. | Stat. | Stat. | SE | Stat. | SE 
Writing 278 I 24 | 12.09 | 3.7 | .26 AS 1.1 29 
Competence t0 
Writing 275 3 24 | 12.84 | 3.7 | -12 | .15 .30 29 
Competence t1 
Writing 276 3 23 | 13.36 | 3.9 | -16 | .15 | .25 | .30 
Competence t2 


For a comparison of scores on the Writing Competence scale between the 
experimental and the control groups over time, a mixed between-within 
subject analysis of variance was conducted. The results show a significant 
main effect for time, F (2,222) = 16.005, p = .000, n? partial = .07, with both 
groups of students showing an increase in the Writing Competence scores 
across the three points of time (see Fig. 2). This main effect, when compar- 
ing the two groups of students, was significant with F (1, 222) = 23.169, 
p = .000, 1 partial = .10, suggesting a difference in Writing Competence 
scores between the experimental and the control groups. The effect size is 
moderate (see Fig. 2). 
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Fig. 2: Intervention effects on Writing Competence 


A one-way repeated measures ANOVA conducted for each group shows 
significant differences in the Writing Competence scale values of the experi- 
mental group over time (F (2, 118) = 20.956, p = .000) and no change in 
the Writing Competence of the control group (F (2, 103) = 2.567, p > .05). 
Follow-up comparisons (Bonferroni) for the experimental group show a 
significant increase in scores not only between the first (M: 12.90, SD 3.8) 
and the second (M: 14.25, SD: 3.60) point of time, but also across the first 
and the third (M: 14.50, SD: 3.58). This suggests that the didactic concept 
had a positive impact on VET students’ writing competence and provides 
evidence of a long-term retention of the learning effect. 

Fig. 2 also shows that the experimental and control groups were not com- 
parable regarding their pre-treatment writing ability (EG M: 12.90, SD 3.88 
vs. CGM: 11.50, SD: 3.54). For this reason, but also because pre-treatment 
ability can be a basic or even the single predictor for post-treatment ability 
(Vanhove 2015), we used the pre-test scores as covariates. The results of the 
analysis of covariance (ANCOVA) indicate a strong relationship between 
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the pre-intervention scores and post-intervention scores on Writing Compe- 
tence (F (1, 222) = 228.632, p = .000, n? partial = .51). However, the group 
effect remains significant (F = (1,222) = 25.200, p = .000, n? partial = .10).? 
The quality of a writing test can also be measured by its impact on the 
individuals involved (especially test-takers and teachers) and the immedi- 
ate educational system (Weigle 2002: 53). The dissemination of the study 
results was, therefore, particularly important. After the study, we gave 
individual feedback to the participating students on the development of 
their writing competence over time. Additionally, we organized a confer- 
ence with the participating teachers, stakeholders and experts in order to 
evaluate the project. The results were beneficial to all involved and led to 
recommendations for a revision of the writing curricula in VET schools. 


4 Conclusion and recommendations 


Issues of assessment can teach us many things about our subjects and the 
educational contexts. The reliability scores of our measurement procedures 
show that the kind of literacy we aimed to study appeared to be a consist- 
ent construct when relevant scales were developed, as was the case with 
our six subscales of Correctness, Style, Formal Conventions, Structure/ 
Coherence, Content and Communicative Impact, which can be assigned 
to the three competence fields Linguistic Competence, Genre Competence 
and Pragmatic Competence. 

Factor analysis has also shown that these scales may be seen as dimen- 
sions of a strong main factor, which we interpret as ‘general writing com- 
petence’ and which gives the measured construct consistency and internal 
validity. The validity of our scales is also supported by the results regard- 
ing the impact of the intervention. The change in the teaching of writing 
was reflected in higher scores for writing competence. We may, therefore, 
conclude with confidence that the scales developed are sensitive to different 
modes of teaching and enable differentiation among writing performances 
even at a very basic skills level. 

Respecting the specificity of literacy with regard to educational and lin- 
guistic contexts is an essential prerequisite for successful measurement. 


9 For further results, see Hoefele & Konstantinidou (in prep.). 
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Adaptation to context must be reflected in scale definitions and scoring 
procedures. Although we believe that the competence fields (Linguistic 
Competence, Genre Competence and Pragmatic Competence) can be re- 
produced in many contexts, their measurement must be adapted not only 
to the achievement level, but also to the task dimension of the literacy field 
in question with respect to its communicative, textual and educational 
characteristics. The more advanced students are, the easier it should be to 
develop scales with enough variability to discriminate between levels of 
ability simply because texts are longer and skills more differentiated. What 
we were able to show in this study, however, is that discrimination is also 
possible at very basic competence levels. 

From its very beginnings, our study was not primarily aimed at demon- 
strating that process orientation is superior to conventional approaches to 
the teaching of writing, rather, it was designed to address a very practical 
question of teaching in a field that is practically untouched by research. 
Teaching writing in VET schools is often regarded as a field of pedagogy 
where deficits from earlier education prove to be obstacles to substantial 
progress, and where teachers often resign themselves to the many mistakes 
their students make even in very short texts. What we have shown is that 
progress is possible and that a close look at literacy development in this 
group demonstrates subtle differences in the kinds of skills that literacy 
comprises at this early stage and in this particular field of education. 

Our research has been highly appreciated by both teachers and admin- 
istrators. In addition to our cooperative approach (creating the writing 
assignments together with teachers), one of the main reasons for this ac- 
ceptance was the assessment methodology that resembles examination situ- 
ations in school (the test situation) and uses rating scales which can pick 
up relevant dimensions in any kind of text production. The scales allow 
teachers to understand where deficits in literacy development are located 
and which types of linguistic, genre-related and pragmatic competence are 
missing. For this reason, our measurement approach facilitates the improve- 
ment of educational practice and the introduction of change in the teach- 
ing of writing on the basis of evidence which, by teachers, is considered 
consistent and well-justified. 
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Writing competence profiles 
as an assessment grid? — Students’ L1 and L2 
writing competences and their development 
after one semester of instruction 


Abstract 

English: For the design of compensatory writing courses and determining the pro- 
gress that students make after one semester of writing instruction, the L1 (German) 
and L2 (English) writing skills that German students come with when entering 
university were assessed using argumentative essays they had to compose at the 
beginning of writing courses and at their end respectively in either their L1 (n=26) 
or their L2 (n=35) depending on the type of writing course (German or English) 
they attended. The essays were evaluated by three raters in a discursive consensual 
assessment procedure using a (text-)linguistic error classification scheme as well 
as a holistic evaluation of their argumentative rigour. The application of the error 
classification scheme yielded assessments of the texts on four levels (formal, lexical, 
grammatical and textlinguistic correctness), whereas the holistic evaluation yielded 
an assessment of the texts’ argumentative rigour (stringency). A comparison of the 
results obtained at the beginning and at the end of the courses showed significant 
improvements at all levels for the English texts (L2) but only at the lexical level 
and the text level for the German texts. Furthermore, significant correlations were 
found between the lexical and grammatical sub-competencies, between the lexical 
and textlinguistic sub-competencies and between the grammatical and textlinguistic 
sub-competencies within each language at least at the beginning of the courses. 
Comparisons between languages both at the beginning and at the end of the courses 
revealed that L2 text production competence significantly lagged behind L1 text 
production competence at all levels except for the text level at the beginning of 
the writing courses. These findings are interpreted against the background of the 
cognitive capacity theory of writing and dynamic systems theory. The visualiza- 
tion of the results in area graphs representing writing competence profiles revealed 
characteristic patterns which suggest that such area charts, when available for a 
larger spectrum of writing competences, may be usable as grids for the assessment 
of writing competence. 
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German: Zur Konzeption kompensatorischer Schreibkurse fiir die L1 Deutsch und 
die L2 Englisch sowie zur Ermittlung der Schreibkompetenzentwicklung nach einem 
Semester Unterricht wurden die L1- und die L2-Schreibkompetenz von deutschen 
Studierenden anhand von argumentativen Texten erhoben, die diese zu Semesterbe- 
ginn und am Semesterende in Abhangigkeit vom besuchten Schreibkurstyp (Deutsch 
oder Englisch) in ihrer L1 (n=26) bzw. ihrer L2 (n=35) verfassen mussten. Die Texte 
wurden von drei Lehrkräften in einem diskursiven Prozess analytisch mit Hilfe 
eines (text-)linguistischen Fehlerklassifizierungsrasters sowie holistisch bewertet. 
Die Anwendung des Fehlerklassifizierungsrasters ergab eine Bewertung auf vier 
Ebenen (formale, lexikalische, grammatikalische und textlinguistische Korrektheit), 
während die holistische Evaluation zur Beurteilung der argumentativen Stringenz 
genutzt wurde. Der Vergleich der zu Kursbeginn und am Kursende erzielten Re- 
sultate ergab eine signifikante Verbesserung in allen Bereichen bei den englischen 
Texten (L2), jedoch nur eine Verbesserung im lexikalischen und textlinguistischen 
Bereich bei den deutschen Texten (L1). Darüber hinaus fanden sich innerhalb der 
Sprachen zumindest zu Kursbeginn signifikante Korrelationen zwischen der lexika- 
lischen und grammatikalischen Teilkompetenz, zwischen der lexikalischen und der 
textlinguistischen Teilkompetenz sowie zwischen der grammatikalischen und der 
textlinguistischen Teilkompetenz. Der Vergleich der Ergebnisse in beiden Sprachen 
zeigte sowohl in der Erhebung zu Kursbeginn als auch am Kursende, dass die Text- 
produktionskompetenz in der L2 in allen Bereichen geringer ausfiel als in der L1; 
eine Ausnahme bildete lediglich zu Beginn der Kurse die textlinguistische Teilkom- 
petenz. Die Ergebnisse werden vor dem Hintergrund der Cognitive Capacity-Theorie 
und der Dynamic Systems-Theorie interpretiert und in Flächendiagrammen, die 
Schreibkompetenzprofile darstellen, visualisiert. Diese Visualisierung lässt charakte- 
ristische Muster erkennen, die nahelegen, dass solche Flächendiagramme als Raster 
zur Bewertung von Schreibkompetenz genutzt werden könnten, wenn sie für ein 
größeres Spektrum von Schreibkompetenzprofilen vorliegen. 


1 The context and rationale of the study 


In the wake of the Bologna reform, German universities have realized that 
more time and effort need to be devoted to the development of students’ 
literacy or text competence, encompassing text reception competence, text 
production competence and the competence to learn from texts (Portmann- 
Tselikas/Schmölzer-Eibinger 2008: 5). At least three reasons have led to 
this insight: First, the development of these competencies is still in progress 
when students enter university and needs to be fostered actively in its dis- 
Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 


Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Writing competence profiles as an assessment grid? 105 


cipline- and task-specific continuation (see, e.g., Beaufort 2007). Second, 
shorter university degree programmes give students less time to develop 
these competencies as a ‘by-product’ of their academic socialization. And 
third, an increasing student intake at universities (more than 50% of an 
age cohort in Germany in 2012) has led to more heterogeneous entrance 
qualifications among students due to more diverse educational biographies 
and thus the need for compensatory courses including compensatory writ- 
ing courses (see also Göpferich 2015). 

In order to gear its compensatory writing courses to the actual needs of 
the students who attend them, a research project was launched in 2012 at 
the newly established writing centre of the University of Giessen (Germany) 
in which students’ writing skills in their L1 (German) and their L2 (English) 
were assessed in areas that have frequently been assumed as mastered at 
university entrance level. Based on the findings of this assessment, L1 and 
L2 compensatory writing course syllabi were designed which address the 
needs detected, and students’ progress was assessed after one semester of 
instruction (30 contact hours and 30 hours of homework) in such L1 and 
L2 compensatory writing courses. 

The assessment results were visualized in area graphs, so-called writing 
competence profiles, which set the individual writing sub-competencies 
analysed in the study in relation to each other. A comparison of these area 
graphs from the beginning and from the end of the writing courses and 
between languages illustrates the progress students made in the course of 
one semester of instruction and shows the developmental lag between the 
L1 and the L2 writing skills. 

The compensatory writing courses focused on in this study were intro- 
duced against the background that basic linguistic skills related to writing, 
such as writing in a formally, lexically and grammatically correct and coher- 
ent manner (for details of the construct of writing competence measured, 
see Section 3.1), cannot be remedied in discipline-specific writing courses 
and therefore should be addressed in introductory general writing courses 
preceding discipline-specific courses for the following three reasons. First, 
dealing with such linguistic problems in discipline-specific writing courses 
would offer no challenge for those students who have already acquired 
the relevant competencies; second, teachers of discipline-specific writing- 


intensive seminars are subject-domain specialists and cannot be expected 
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to have the specific linguistic and didactical knowledge to address general 
language-related issues; and third, discipline-specific writing courses ideally 
should be devoted to the advanced stages of academic writing socialization. 
In this manner, domain-specific literacy development and the development 
of domain-specific knowledge and competences can benefit from each other 
(cf. Ulmi et al. 2014: 7, 226). 


2 Theoretical background and previous studies 


This section introduces the models of writing competence development 
against which the findings from the present study will be interpreted, and 
sets this study in relation to previous ones comparing L1 and L2 writing 


skills. 


2.1 Modelling writing competence development 


Many models and theories of writing competence development that have 
been suggested to date have their origins in cognitive psychology. They 
explain processes of writing competence development with reference to 
automatization or routinization of sub-processes, or lower-order processes, 
of writing, which is assumed to release working memory capacity with the 
ensuing possibility of a reallocation of these cognitive resources to higher- 
order processes. Among these theories are McCutchen’s (1996) cognitive ca- 
pacity theory of writing and Kellogg’s (2008) macro-stages of writing skills 
development, which are compatible with each other. According to Kellogg’s 
macro-stage model of writing skills development, the cognitive capacity 
that becomes available through increasing automatization of lower-order 
processes in accordance with the cognitive capacity theory can be used by 
authors, for example, to take an increasing number of perspectives on their 
texts into account. Whereas in the early stage of knowledge-telling, authors 
are limited to their author’s perspective (i.e., what the author intends to 
express, the author meaning), the stage of knowledge-transforming is char- 
acterized by authors being able to keep both the author meaning and the 
actual text meaning in mind and compare them for discrepancies, which 
can then be corrected. Kellogg’s highest stage of writing skills development, 
knowledge-crafting, then allows authors to keep both their author meaning 
and the text meaning in mind and, in addition, take into account whether 
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their prospective readers will be able to derive the intended meaning from 
the text. This allows them to take a more reader-centred perspective. 
Steinhoff (2007) and Pohl (2007) conducted corpus-linguistic studies to 
analyse the development of students’ academic writing competence. Stein- 
hoff, for example, compared academic textual routines or procedures! used 
in German term papers composed by university students at different stages 
of their academic socialization with their equivalents in articles composed 
by scholars and functionally comparable routines and formulations in a 
corpus of journalistic texts. For the interpretation of his data, he drew on 
Piaget’s theory of cognitive development. According to this theory, the 
driving force in the development of intelligence is the reestablishment of an 
equilibrium, whenever this has been destroyed, by processes of assimilation, 
i.e., the adaptation of the environment to the learner’s cognitive schemata, 
or accommodation, i.e., the adaptation of the learner’s cognitive schemata 
to the environment (Piaget 1972: 191; cf. Ortner 1995: 336). With regard to 
academic writing competence development, the search for this equilibrium 
means developing writing behaviour that is adequate in academic contexts 
out of writing behaviour that was acquired for other contexts and is found 
to no longer fulfil its function in a satisfactory manner in the new context. 
To determine students’ university entrance qualifications in writing, it is 
exactly the writing behaviour that students acquired for other contexts 
which needs to be focused on and thus is the focal point of the present study. 
In his corpus analyses, Steinhoff (2007: 419) found that the develop- 
ment of students’ academic writing competence follows an inter-individual 
pattern. At the early stages of their academic writing skills development, 
assimilation processes dominate, i.e., processes in which known behaviour 
is used to solve new problems, whereas at more advanced stages, accom- 
modation processes take over, i.e., processes in which new strategies are 
made use of. At the initial stage, two types of problem solving were found 
to dominate: transposition and imitation. Transposition is an assimilating 
strategy, in which writers try to solve their problems of academic writing by 
resorting to means and measures that are untypical of academic writing and 


1 The term textual routines/procedures (“wissenschaftliche Textprozeduren”) 
refers to cognitive acts or moves, which may be closely connected with specific 
formulations (cf. Steinhoff 2007: 431). 
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occur in genres they are more familiar with, for example, in journalistic (or 
popular-science) texts, in genres that students were required to compose at 
school, and in oral communication. Imitation is an accommodating strat- 
egy in which students, unsuccessfully or only partially successfully, try to 
imitate academic language. Formulations that result from this strategy are 
characterized by, for example, extreme complexity, excessive nominaliza- 
tions and the use of an extremely formal style, which often results in a lack 
of comprehensibility (Steinhoff 2007: 143 ff., 423). The stages of transpo- 
sition and imitation are followed by a stage of transformation, in which, 
in an accommodating manner, the repertoire of academic formulations 
that students have at their disposal is expanded. Students begin to learn 
typical formulations of academic writing and the communicative functions 
for which they are used in academic contexts. The fact that, at this stage, 
they have not yet fully reached expert competence, shows in, for example, 
wrong collocations and an excessive and repetitive use of certain domain- 
specific formulations. Another characteristic of this stage is that students 
begin to recognize that certain general-language expressions are used for 
more specific purposes in academic texts (Steinhoff 2007: 146 ff., 423 f.). 
Students’ writing competence development reaches its (preliminary) final 
stage, the stage of contextual adequacy, when they start to use academic 
language following domain-specific conventions and employing a differenti- 
ated academic vocabulary (Steinhoff 2007: 424, cf. 148 f.). 

Another paradigm in which writing competence development can be 
modelled is dynamic systems theory (DST; Thelen/Smith 1994; Van Gelder 
1989).2 DST has found its way from mathematics into various other fields 
of research, such as developmental psychology (see Thelen/Smith 1994), 
second-language acquisition (e.g., de Bot/Lowie/Verspoor 2007; Verspoor/ 
Lowie/Van Dijk 2008), translation studies (Göpferich 2013) and also L2 
writing skills development (e.g., Verspoor/de Bot/Lowie 2004; Verspoor/ 
Schmid/Xu 2012; Nitta/Baba 2014). In DST, competences or skills are en- 
visaged as dynamic systems, i.e., sets of variables that are interconnected 
and thus interact over time (de Bot/Lowie/Verspoor 2007: 8). With regard 
to writing competence, these variables can be regarded as variables for writ- 


2 The outline of DST presented here is based on an earlier version in Göpferich 
(2013) introducing DST in the context of translation competence development. 
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ing sub-competencies that, in their entirety, make up writing competence. 
These individual sub-competencies may not develop at the same pace, nor 
will they always develop in a linear manner. Some may stagnate while 
others continue to develop. Certain sub-systems (i.e., sub-competencies or 
clusters of sub-competencies) may be precursors of other sub-systems in the 
developmental process. For a specific sub-competence to start developing, 
it may be necessary for other sub-competencies to have exceeded a certain 
threshold value. If we start from the cognitive capacity theory of writing 
(McCutchen 1996), which assumes limited working memory capacity, it ap- 
pears plausible to assume that the value one variable takes, i.e., the cognitive 
effort required for a specific sub-process of writing, such as lexical choices, 
has an effect on all the other variables. For example, if one sub-competence 
reaches an advanced level, which may result in automatized performance 
of the respective tasks, working memory capacity is released. This capac- 
ity then becomes available for the application of other sub-competencies 
and their development, such as the capacity to take larger stretches of a 
text into account to create coherence, for which there may not have been 
enough cognitive resources left as long as other sub-processes still needed 
cognitive effort. 

DST further assumes that, in dynamic systems, repeller states and at- 
tractor states exist. These can best be explained by means of the metaphor 
of a plane with holes and bumps in it on which a ball rolls. The ball and 
its trajectory on the plane represent a specific competence and its develop- 
ment path. On its ways over the plane, the ball is attracted by the holes 
(attractor states) and repelled by the bumps (repeller states). To get it out 
of a hole, much energy is needed. Such holes can explain fossilization of 
certain errors or stages of development, as observed in second-language 
acquisition, which can only be overcome by intensive training. The develop- 
ment that a particular person undergoes, i.e., the trajectory of the ball, may 
be highly individual, yet attractor states account for the fact that certain 
sub-competencies seem to occur in a specific order or at a specific stage of 
competence development because the corresponding attractor states, in an 
evolutionary perspective, possess qualities that make the application of 
cognitive resources settle into an equilibrium at these stages. In DST, the 
fact that a certain competence stage has been achieved becomes visible by 


a relative stagnation in the development of the values the various variables 
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that make up the system take at this stage, whereas a move out of the hole, 
a new competence development burst, shows in a large variability in the set 
of variables which form the competence system (Thelen/Smith 1994: 97). 

What complicates the analysis of writing skills development in a DST 
approach is that dynamic systems are nested, i.e., every system is always 
part of a larger system. Accordingly, writing competence forms a sub- 
system of the larger system of communicative competence among others. 
The aspects of writing competence focused on in this article (for details, see 
Section 3.1) again form only one component of the larger academic writing 
competence system analysed by Steinhoff and Pohl. In their investigations, 
however, they concentrated on the developmental stages following a writ- 
ing competence level that students are generally assumed to have acquired 
already at school, which, as the findings from the present study will show, 
is not always the case. 

An even more encompassing concept of writing competence can be found 
in recent publications on writing competence assessment that draw on Bron- 
fenbrenner & Morris’ (2006) bioecological model of human development. 
These publications (e.g., Slomp 2012; Driscoll/Wells 2012; Wardle/Roozen 
2012) plead for analyses of writing competence development which do not 
only take into account contextual factors such as the type of writing instruc- 
tion students were exposed to, but also characteristics of the individual, 
dispositions, such as value, self-efficacy, attribution and self-regulation 
(Driscoll/Wells 2012: 7). This nesting or embeddedness of competences 
makes it difficult to draw a border around the system an investigation fo- 
cuses on, and drawing such a border always means ignoring factors beyond 
the border that may have effects relevant to the system in question. 


2.2 Language (in-)dependence of writing skills 


The main difference that has been found between L1 and L2 text production 
is the higher cognitive demand involved in choices of lexis and grammar in 
the L2 as compared with the L1. In accordance with the cognitive capacity 
theory of writing, however, higher cognitive demands at these lower levels 
of text production have been found to have negative effects on higher-level 
decisions (Silva 1992; Devine/Railey/Bischoff 1993; Whalen/Menard 1995; 
Schoonen et al. 2003; Roca de Larios/Manchon/Murphy 2006; see also the 
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literature review by Cumming 2001). Such findings provide support for 
the assumption that the epistemic benefits of writing are less pronounced 
when this writing takes place in the L2. They also warrant the assumption 
that the epistemic function of writing can only be fully exploited in the L1 
if students have achieved a certain minimum fluency with regard to lower- 
order processes, e.g., at the lexical and grammatical levels, when writing 
in this language. This underlines the importance of offering compensatory 
writing courses which address these issues for preparing students to derive 
the maximum benefit from discipline-specific writing courses where writing 
is also used as a means of more profound reflection. 


3 Methodology 
3.1 The construct assessed and the assignments 


The construct of writing competence assessed in the present study was 
the students’ ability to express themselves in a formally and linguistically 
correct, cohesive, coherent and well-reasoned manner in a genre they were 
familiar with from their secondary education (for the reasons, see Section 
2.1). Requiring them to compose texts of unfamiliar genres could have led 
to a lack of motivation and, according to the cognitive capacity theory of 
writing (McCutchen 1996), to cognitive capacity being absorbed by reflec- 
tions concerning the unfamiliar genre which could then have led to poorer 
performance on lower levels, such as the levels of formal, lexical and gram- 
matical correctness, that would otherwise have received more attention. 
In such cases, performance on these lower levels might not have reflected 
students’ real competence at these levels, although their competence was 
more relevant for the purposes of this study than their performance. Testing 
students’ competencies at these levels in tests which did not require them 
to compose a text but to fill in gaps or complete sentences was not an op- 
tion either because in that case students could have fully concentrated on 
the level in focus, which is an unrealistic situation because communication 
usually takes the form of texts, and composing texts involves more complex 
cognitive processes which — at least to some extent — have to be juggled 
in parallel. Furthermore, assignments had to be selected that did not re- 
quire specialized prior content knowledge that would have led to failure if 
students had not possessed it. This was a requirement because neither the 
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students’ prior content knowledge nor their ability to do literature research 
were part of the construct for which the students were tested, and failure 
to complete the assignment because of a lack of prior knowledge would 
not have allowed to elicit data on those aspects of writing skills that were 
in the focus of this study. In addition, the texts to be composed had to ad- 
dress a readership whose prior knowledge and interests could be judged 
realistically by the students. 

For these reasons, the particular type of assignment selected was writing 
an argumentative article for a student magazine. This genre could be ex- 
pected to be familiar to students because the composition of argumentative 
texts forms part of the German standard school curriculum. Furthermore, 
the ability to compose a logical argumentation is a central academic skill. 
The manner in which students connect statements indicates whether they 
are able to keep their argumentative goal in mind and come to a logical 
conclusion. To ensure that students did not lack the prior knowledge rel- 
evant for completing the assignment, topics were selected about which all 
students could be expected to have the relevant prior knowledge required. 
Offering them different topics to choose from was an additional measure to 
make sure that the topic did not present an obstacle for students. 


3.2 Data collection 


In their second weekly 90-minutes writing course sessions (out of 15), the 
participants of general writing courses that were offered at the writing cen- 
tre of Justus Liebig University in Giessen in the winter semester 2012/13, the 
summer semester 2013 and the winter semester 2013/14 were required to 
write an argumentative text of approximately 250 to 350 words on one of 
three topics they could choose from for a student magazine (see the assign- 
ments in Appendix A). Students who attended writing courses for German 
had to compose their texts in their L1 German; students who attended 
writing courses for English had to do so in their L2 English. They were not 
allowed to use external resources. In the last session of their writing courses, 
students were required to compose another text of the same genre and in 
the same language on one of three other topics they could again choose 
from. The texts had to be completed within the 90-minutes sessions. The 
students wrote the texts in a computer lab using Microsoft Word as the 


word processor they were familiar with. 
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3.3 Research population 


The population of the study presented in this article was composed of two 
groups: The first group comprised 49 students who attended writing courses 
in their L1 German (German group). The second group comprised 51 other 
students who attended writing courses in their L2 English (English group). 
Out of the 49 students of the German group, only 26 completed their writ- 
ing assignments at the end of the German writing courses; out of the 51 
students of the English group, only 35 did so in the English courses. Only 
the data obtained from students who completed the writing assignments at 
both the beginning and the end of their writing courses could be taken into 
account in the comparisons of their performances at the beginning and at 
the end of the courses. 

The 49 participants who attended the German writing courses came 
from all faculties. 35 of them were enrolled in BA programmes, 6 students 
in MA programmes, 6 students in teacher training programmes, 1 student 
in an 8-semester university diploma programme and another student in a 
programme for veterinary medicine. They were in their 1st to 22nd semes- 
ters (median semester: 3; average semester: 4). 

Out of the 51 participants of the English writing courses, 24 were enrolled 
in study programmes with English language and literature as one of their 
subjects (21 students in BA programmes, 1 student in an MA programme 
and 2 students in a teacher training programme). The remaining 27 students 
came from all faculties. 27 out of the 51 students were in their 2nd semester, 
3 in their 4th semester, 4 in their 5th semester, 4 in their 6th semester, and 13 
in a higher semester (median semester: 2; average semester: 3.4). 


3.4 Treatment 


In the German writing courses, the following topics, which address both 
the weaknesses detected in the texts composed at the beginning of the 
courses and additional topics, were covered: planning writing projects, time 
management; finding a topic and specifying a research question; literature 
research and reading strategies; strategies for overcoming writer’s block; 
writing an outline; text organization; planning one’s line of argumentation; 
specificities of academic writing (in contrast to non-academic writing); pro- 
totypical text macrostructures and typical formulations for specific sections 
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(scaffolding); text coherence and cohesion, advance organizers and logical 
connectors; register, readability and comprehensibility; systematic revision, 
peer feedback; grammar: subjunctive, active and passive voice; punctuation 
rules; and correcting spelling and punctuation errors. 

For English, two types of writing courses were offered. One type followed 
an ‘English only’ approach, the other one, a contrastive German-English ap- 
proach. In the latter type of course, students were not only taught, and had to 
complete assignments, in English but were also confronted with genres in Ger- 
man which they had to compare with their English counterparts and to translate 
into English. Both types of courses covered the following topics: brainstorming; 
specificities of academic writing in English, useful vocabulary and formula- 
tions (scaffolding); Fillmore’s scenes-and-frames semantics and its relevance for 
academic writing; finding a topic and specifying a research question; planning 
and writing argumentative texts with a focus on their macrostructure; how 
to quote and avoid plagiarism; paragraph structure (topic sentence, introduc- 
tion, supporting details, conclusion/transitions); text coherence and cohesion 
with a focus on logical connectors, parallelism; introduction to different gen- 
res: reading and composing book reviews, instructions (‘how to’-texts) and 
popular-science texts; taking one’s audience into account; punctuation rules; 
refreshing grammatical knowledge: use of articles, prepositions, syntax, func- 
tional sentence perspective; editing. In addition, the contrastive German-English 
course also introduced students to Fillmore’s scenes-and-frames semantics and 
its relevance for translation; criteria for translation quality assessment; using 
monolingual and bilingual dictionaries; and translating book reviews. 

The diagrams that follow indicate for all students whether they were stu- 
dents with English as a major or minor in their degree programme (“Eng”) 
or not (“non_Eng”). For those who had English as a major or minor in their 
degree programmes, they also indicate whether they attended a course that 
was taught in English only (“Eng_mono”) or a contrastive German-English 
course (“Eng Ger”). 


3.5 Data analysis 


The texts produced at the beginning and at the end of the courses were 
subjected to in-depth analyses, which combine a detailed error analysis with 
a holistic assessment (cf. Weigle 2002). For the error analysis, the primar- 


ily (text-)linguistic error classification scheme in Table 1 was used. Earlier 
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versions of this error classification scheme, which were slightly adapted for 
the present study, had been developed on the basis of linguistic errors that 
occurred in German popular-science articles produced by students of tech- 
nical communication (cf. Gopferich 2002: 286 ff.) and translation errors 


made by translation students when translating from their L2 (English) into 
their L1 (German) (Göpferich 2010: 54 f.). 


Table 1: Error classification scheme 


Error category 


Description/Example 


Formal errors 


punctuation 


missing or wrong punctuation mark; if both a 
comma at the beginning and at the end of an 
insertion are missing, this is counted as only 
one error; repeated comma errors are counted 
as repetition errors only if they have the same 
cause 


spelling 


spelling mistake which is not an obvious typo 


formatting 


line break where there should be none (the 
participants were not required to do any other 
formatting in the text); italics where there 
should be none and vice versa 


Lexical errors 


semantic 


use of words and phrases which do not express 
the intended meaning either denotatively 
(semantic: denotation) or connotatively 
(semantic: connotation). The category 
“semantic: connotation” includes the use of a 
wrong register at word level. 

use of expressions which do not exist; use of 
word forms from another language which 
cannot be expected to be understood in the 
target culture, etc. 

Note: This only refers to words and phrases, 
not to longer stretches of text that have been 
translated literally and are unidiomatic in the 
target language. See: “idiomaticity/genre”. 
Note: Blendings and wrong illocutionary 
indicators are classified as separate categories. 
See also “collocation” and “modality/ 
illocution”. 
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Description/Example 


collocation 


wrong collocation (e.g., fast speed instead of 
high speed). This category only includes cases 

in which the meanings of the words used are 
appropriate but in which these words cannot be 
combined for other reasons, such as idiomaticity 
or convention. 


blending 


error caused by melding together parts of 
linguistic units or constructions which enter 
working memory simultaneously 


preposition 


use of a wrong preposition (see also “other 
grammar”) 


modality/illocution 


wrong illocutionary indicator, such as sollte 
(recommendation) instead of muss (instruction) 


redundancy 


superfluous repetition of meaning components, 
ideas, statements or words including tautologies 
(see also “text coherence”) 


Grammatical errors 


tense 


use of wrong tense 


aspect 


case, number, agreement 


use of wrong aspect 


use of wrong case or grammatical number, 
mostly after prepositions or in appositions; 
agreement error 


mood 


wrong mood, e.g., in indirect speech 


voice 


active voice instead of passive voice and vice 
versa; e.g. Die Truppenstärken erhöhten sich. 
(which is wrong because they cannot do that 
by themselves) instead of Die Truppenstärken 
wurden erhöht. 


word form 


morphologically wrong word form, such as 
adjectives instead of adverbs and vice versa; 
wrong formation of past tense forms. 
Where in doubt whether “word form” or 
“semantic: denotation” should be used, 
“semantic: denotation” was used if the error 
could be assumed to be due to deficiencies 
in the lexicon, and “word form”, if the error 
was rather caused by a lack of grammatical 
competence. 
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Error category 


Description/Example 


syntax 


syntactic error; participle instead of 

infinitive and vice versa; missing participle 

in a construction where German just uses a 
prepositional phrase without participle; long 
prenominal modifiers in English (interference 
from German) which should be transformed 
into a participle construction after the noun; 
constructions which are hard to understand due 
to their length, long parentheses, etc.; utterances 
which are grammatically correct but which 
would only make sense if additional lexical 
elements were inserted, e.g., The few utterances 
could not be categorized as either one of the two 
models. Comment: For this utterance to make 
sense it would have to be reformulated into: 
The few utterances could not be categorized as 
belonging to either one of the two models. 

lack of parallelism at the syntactical level 


valency 


specifier (article or determiner) 


missing actant; use of a lexical element that 
requires further specification; applicable also to 
nominalised verbs, e.g., Aufenthalt (from sich 
aufhalten in), where the indication of a place is 
required 


use of a determiner, e.g., an article, where there 
should be none; use of a definite article where an 
indefinite article should be used; etc. 


infinitive/participle 


grammatically wrong use of an infinitive or 
participle construction (e.g., Das Wetter war zu 
schlecht, um schwimmen zu gehen.) 


secondary subjectivization 


use of verbs expressing human actions in 
connection with non-human subjects (possible 
in English but rare in German); e.g., This book 
describes (correct) vs. Dieses Buch beschreibt 
(wrong). 


other grammar 


other grammatical errors, such as the use of a 
prepositional phrase instead of a genitive and 
vice versa (e.g., von seinem Vater instead of 
seines Vaters) 
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Error category 


Description/Example 


Text-level (textlinguistic) errors 


text coherence 


incoherent text segments, e.g., logically wrong 
connection of clauses and sentences by the use 
of semantically inappropriate conjunctions; use 
of wrong pronouns; sentence not related to its 
context; referent unclear; missing second part of 
correlative (two-part) conjunctions; repetition 
of a noun phrase where a pronoun would be 
sufficient 


sense incomprehensible or nonsensical section longer 
than a phrase (otherwise it is counted as a 
semantic error), contradictory statements 

implicitness too much information left implicit, e.g., 


author does not express something to which 
a conjunction, etc. refers (e.g., There are three 
types of birch trees. Therefore, I will describe 
only one. Here, therefore refers to a sentence 
that was left implicit, i.e., I cannot cover them 


all.) 


functional sentence perspective 
(FSP) 


wrong topic-comment structure (theme/rheme) 


rhetoric loss of communicative emphasis or effect (e.g., 
replacing a poem by a mere description of 
its content); literal repetitions (see, however, 
“redundancy” and “text coherence”) 

Other 

idiomaticity/genre unidiomatic expression which does not lead to a 


change of meaning but may make the text hard 
to understand and betray that it is a translation 
in a negative sense; use of expressions which 

do not conform to genre conventions (e.g., Das 
Bild ist kein Zufallstreffer. instead of Das Bild ist 
kein Schnappschuss. and Anfangend mit Namen 
as a title.) 

Note: If this refers to lexical units and not to 
longer stretches of text, “semantic: denotation” 
was used. 


cultural specificity 


missing adaptation to the target culture or 
missing cultural neutralisation 
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Every error identified in the texts was highlighted and annotated with a 
corresponding error tag (see two examples of annotated texts in Appendix 
B). To reduce subjectivity, each text was marked by three raters in a dis- 
cursive process in which discrepancies among the raters were discussed and 
reconciled and the error classification scheme with its explanations specified 
to ensure consistent interpretation where necessary. In order to be able to 
compare the numbers of errors in the texts, which differed in length, the 
number of errors per 100 words (error ratio) was calculated for each text. 

Since text quality cannot be assessed exhaustively on the basis of error ratios 
alone, each text was additionally subject to a holistic assessment in which its 
argumentative rigour (henceforth called stringency) was determined by the same 
three raters who performed the error analysis, again in a process of discursive 
consensual assessment. Stringency is considered a central quality criterion for 
argumentative texts. Stringency assessments focus on whether the line of ar- 
gumentation in a text is logical, whether the text has a logical conclusion and 
whether the line of argumentation leads to this conclusion without deviations. 
For stringency, the texts were rated on a scale from 0 to 3 points. Three points 
were awarded for a high level of stringency without deviations and with a logi- 
cal conclusion; 2 points, for a line of argumentation with few or minor devia- 
tions and a logical conclusion; 1 point, for a text with frequent or substantial 
deviations but a conclusion; and 0 points, for an unorganized text and/or a text 
without any clear conclusion. The two texts in Appendix B are examples of texts 
with minimal stringency (0 points) and high stringency (3 points), respectively. 

In order to obtain a single overall text quality score for each text, the error 
ratios (errors per 100 words) that occurred in each rubric of the error classifi- 
cation scheme were converted into points, too, again on a rating scale from 0 
to 3 for each rubric so that an overall score could be calculated by adding the 
scores in each of the rubrics. Three points were awarded for extremely low 
error ratios in the respective rubric, 0 points, for extremely high error ratios. 
Errors of the rubric “Other” occurred only extremely rarely, so this rubric 
could be ignored, leaving the four rubrics “formal correctness”, “lexical cor- 
rectness”, “grammatical correctness” and “textlinguistic correctness”. The 
lattermost rubric represents the text-microstructural counterpart of the cat- 
egory “stringency”, which takes text-macrostructural aspects into account. 

When converting the error ratios into points, potential outliers had to be 
excluded. This was achieved as follows: For the error ratios the students had in 
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each of the rubrics indicated above, the medians and standard deviations were 
calculated. For each rubric, the sum of the median error ratio and the standard 
deviation (and not the highest error ratio that had occurred) was divided by 
four. The result indicates the length of each of the four stretches of equal length 
(quartiles) into which the range of error ratios from 0 to the sum of the median 
and the standard deviation can be subdivided. The error ratios in the lowest 
error ratio quartile were converted into 3 points, the error ratios in the second 
lowest error ratio quartile, into 2 points, etc. (see Fig. 1). In this manner, each 
text was assessed according to 5 criteria, for each of which it obtained 0 to 3 
points. These could then be added to obtain a single quality score for each text. 
standard 


deviation 
of error ratios 


highest 


0 errors error ratio 


median 
of error ratios 


3 points 2 points 0 points 


Fig. 1: Conversion of error ratios into points 


In order to allow for a comparison of the results of the German and the Eng- 
lish texts, the quartiles were calculated on the basis of the error ratios obtained 
in all German and English texts composed at the beginning of the semesters 
(n=100), i.e., the error quartiles were calculated on the basis of one median 
and one standard deviation for 100 texts (49 in German and 51 in English) 
in each of the four rubrics “formal errors”, “lexical errors”, “grammatical 
errors” and “text-level errors”. The significance level was set at 10% (p< 0.1). 


4 Results 


Fig. 2 provides an overview of the mean error ratios that occurred in each 
of the rubrics of the error classification scheme in Table 1 in the German 
(L1) and the English (L2) texts composed at the beginning (Pre) and at the 
end (Post) of the writing courses. It also provides insight into the types of 
errors that occurred most frequently in the two languages and thus into the 
topics that were to be covered in the respective writing courses. 
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A comparison of the mean total error ratios in each of the five categories 
“formal errors”, “lexical errors”, “grammatical errors”, “textlinguistic 
errors” and “other errors” at the beginning of the writing courses with 
the respective values at the end of the writing courses reveals that, in both 
languages, these error ratios decreased in all five categories. For the decrease 
in the English writing courses, a Wilcoxon test yielded significant differ- 
ences at a 10% level (p < 0.1) for all error categories, for the decrease in 
the German writing courses, only for the error categories “lexical errors”, 
“text-level errors” and “other errors” (see Table 2). 


Table 2: Statistical significance of decrease in error ratios from beginning (Pre) to 
end (Post) of writing courses (Wilcoxon test, p < 0.1) 


decrease decrease decrease decrease decrease 
formal lexical | grammatical] text-level | other errors 
errors errors errors errors 
German 0.242 0.060* 1.000 0.001* 0.000* 
English 0.057* 0.067* 0.092* 0.004* 0.073* 


*: = significant at 10% level 


Furthermore, the following correlations between error ratios in different 
categories were found (see Table 3). The table only lists those error category 
pairs between which a significant correlation was found in at least one of 
the two languages at at least one point of measurement. 


Table 3: Correlation between error ratios within languages 


lexical | lexical | lexical | lexical] gram- | gram- | formal 
errors— | errors— \errors-|errors-| matical | matical | errors — 
gram- gram- | text- | text- | errors— | errors— | gram- 
matical | matical | level | level | text-level | text-level} matical 
errors errors | errors | errors | errors errors errors 
Pre Post Pre Post Pre Post Post 
German 
— Pearson 0.658* 0.303 |0.529* | 0.192 | 0.546* 0.071 0.699* 
- significance 0.000 0.133 | 0.005 | 0.347 0.003 0.732 0.000 
English 
— Pearson 0.503* 0.583* |0.596* |0.309* | 0.553* 0.490* 0.269 
- significance 0.002 0.000 | 0.000 | 0.071 0.001 0.003 0.119 


*: = significant at 10% level 
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A comparison between languages shows that the mean error ratios in the 
students’ L1 texts (German) were lower in all five error categories (“for- 
mal”, “lexical”, “grammatical”, “text-level” and “other”) at both the be- 
ginning and the end of the writing courses than the corresponding mean 
error ratios in the students’ L2 texts (English). Whereas the mean error 
ratios in the L1 and the L2 differed by less than 1.25 errors per 100 words 
in the rubrics “formal errors” and “other” and even by less than a quarter 
of an error in the rubric “text-level errors”, the differences between the 
L1 and L2 were more substantial in the error categories “lexical errors” 
and “grammatical errors”. On average, the English texts composed at the 
beginning of the semester contained 2.61 more lexical errors and 2.53 
more grammatical errors per 100 words than the German texts composed 
at the beginning of the semester. The English texts composed at the end of 
the semester on average contained 2.49 more lexical errors and 2.14 more 
grammatical errors per 100 words than the German texts composed at the 
end of the semester. A Wilcoxon-Mann-Whitney test yielded significant 
differences at a 10% level (a = 0.1) between the L1 and the L2 in all error 
categories except for text-level errors at the beginning of the semester and 


between all error categories without exception at the end of the semester 
(see Table 4). 


Table 4: Statistical significance of the quality lag in the L2 as compared with the 
L1 (Wilcoxon-Mann-Whitney test) 


formal | lexical | grammatical) text-level | other errors 
errors errors errors errors 
Pre 
-Mann-Whitney-U | 216.500 | 129.000 78.000 424.000 239.500 
- Wilcoxon-W 594,500 | 507.000 | 456.000 802.000 617.500 
-Z -3.635 -4.877 -5.603 -0.689 -4.052 
- significance 0.000* 0.000* 0.000* 0.491 0.000* 
(two-tailed) 
Post 
-Mann-Whitney-U | 256.500 | 106.000 69.500 329.500 0.000 
- Wilcoxon-W 607.500 | 457.000 | 420.500 680.500 630.000 
-Z -2.895 -5.090 -5.623 -1.840 -6.821 
- significance 0.004* 0.000* 0.000* 0.066* 0.000* 
(two-tailed) 


*: = significant at 10% level 
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With regard to stringency, the English texts obtained lower values than 
their German counterparts at the end of the courses (mean stringency of 
the German texts: 0.96 out of 3 at the beginning, 1.92 at the end; mean 
stringency of the English texts: 0.97 at the beginning, 1.07 at the end). The 
improvement from the beginning to the end of the semester turned out to 
be more considerable in German (+0.96) than in English (+0.1). 

The observation of a decrease of error ratios from the beginning to the 
end of the writing courses for both languages and the lower values in Ger- 
man (L1) as compared with English (L2) can also be made with regard to 
the error ratios in most of the individual sub-categories. Exceptions could 
only be found in some of the sub-categories that occurred very rarely and 
therefore will not be further discussed in this article. 

The significant improvement of the students’ results also becomes salient 
in the line charts in Figures 3 and 4, which represent the students’ overall 
text quality scores (in points) at the beginning and at the end of the German 
and the English writing courses respectively. The numbers in parentheses 
indicate the students’ semester of study at the time of data collection. The 
students are arranged according to the total scores they obtained at the 
end of the semester with the best students appearing on the right and the 
poorest on the left. 
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Fig. 3: Students’ writing competence at the beginning and at the end of the 
German (Ger) writing courses represented in points (sum of the points 
for formal, lexical, grammatical and textlinguistic correctness as well as 
stringency) 
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Fig. 4: Students’ writing competence at the beginning and at the end of the 
English writing courses represented in points (sum of formal, lexical, 
grammatical and textlinguistic correctness as well as stringency) 


Figures 3 and 4 show that most students improved (black lines above grey 
ones), whereas only few obtained poorer results at the end than at the 
beginning of writing courses (grey lines above black ones). A compari- 
son between languages shows that the overall scores students achieved in 
their L2 were lower than the scores they obtained in their L1. Except for 
the fourth position from the right, the fourteen best positions in Fig. 4 
are all occupied by students who were taught following the English-only 
(“Eng_mono”) approach. 

Figures 5 to 8 visualize the students’ individual scores in each of the five 
rubrics in relation to each other. In each figure, the students are again ar- 
ranged according to the total scores they obtained at the respective point of 
measurement with the best students on the right and the poorest on the left. 
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Fig. 5: Students’ scores in points for L1 writing (German) in each of the five 
rubrics at the beginning of the semester 


Fig. 6: Students’ scores in points for L1 writing (German) in each of the five 
rubrics at the end of the semester 
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Fig. 7: Students’ scores in points for L2 writing (English) in each of the five 
rubrics at the beginning of the semester 
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Fig. 8: Students’ scores in points for L2 writing (English) in each of the five 
rubrics at the end of the semester 


The flatter graphs for the L2 (English) texts show again that the students’ 
writing competence in their L2 lags behind their writing competence in their 
L1 (German). In the German courses, the weakest students appear to have 
benefited from the courses more than the strongest students. In the English 
courses such a tendency cannot be observed as clearly. For both German 
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and English, the relationships between the scores in the individual rubrics 
seem to become more harmonious with increasing overall results, i.e., the 
higher the students’ overall scores become, the less the scores in the five 
individual rubrics differ from each other and the more likely it becomes 
that the students score in all five rubrics. 

With regard to the development of the students’ scores in each of the 
five rubrics, the hypothesis was set up in Göpferich (2014) that, if students 
had not yet achieved the minimum score of at least one point in each of the 
five sub-categories, their participation in writing courses would first lead 
to an improvement of the lower-level competencies as reflected by “formal 
correctness”, “lexical correctness” and “grammatical correctness”, before 
the higher-level competencies “textlinguistic correctness” and “stringen- 
cy”, which require more working memory capacity, continue to develop. 
Whereas the data obtained for L1 writing competence development seem 
to support this hypothesis, if formal correctness is ignored’, the data ob- 
tained for L2 writing contradict it. In English, a number of students (the 
6 weakest students at the end of the semester) scored in the rubrics “text- 
linguistic correctness” and/or “stringency” without obtaining any points 
in the lower-level categories. In contrast, it is interesting to note, however, 
that Ger_Stud_12, who took a German writing course both in his fourth 
and in his fifth semester mainly improved his writing skills over these two 
semesters with regard to text-level competence and stringency (see Fig. 5, 
where he occurs as Ger_Stud_12a and Ger_Stud_12b). 


5 Discussion 


The error analyses conducted in the present study have revealed that, upon 
entering university (and even in later semesters), many students have not yet 
reached a level of language competence, neither in their L1 nor their L2, that 
has just to be expanded for the specific purposes of academic writing, but 
that they also need instruction to close gaps in their general linguistic and 


3 This can be considered legitimate for two reasons: First, although punctuation 
rules were covered in the courses, they were not focused on. Second, students do 
not attribute much importance to formal correctness and therefore do not invest 
much cognitive capacity into this aspect of text quality, which, as a consequence, 
will then still be available for higher-level decisions. 
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text competence. The error analyses presented in this article have revealed 
areas in which this is the case. 

A comparison of the qualities of the texts produced by the students at the 
beginning of writing courses with those produced by these same students at 
the end of these courses has furthermore shown, for both the German and 
the English courses, that these courses have led to significant improvements 
in the students’ text production competence. The course syllabi thus seem to 
have been effective. What we cannot tell from the data analysed, however, is 
whether modifications in the course syllabi or other teaching methods would 
lead to even better results and whether the effects observed will be lasting ones. 

The findings presented in Figures 7 and 8 might suggest that the “Eng- 
lish only” teaching method could be more effective than the contrastive 
German-English approach. This, however, would be a premature conclusion 
for the following reason. In order to benefit from a contrastive approach, 
students need to develop interference resistance. Interference resistance only 
comes with increasing translation competence and goes hand in hand with 
the ability to overcome fixedness on a source text and to translate more 
creatively (cf. Bayer-Hohenwarter 2012). Translation novices have been 
found to have an inclination to translate word by word and thus to be prone 
to interference, whereas the translation units tackled by expert translators 
tend to be larger and their strategies to be more creative (see, e.g., Gerloff 
1988: 54 ff.; Dragsted 2005; Englund Dimitrova 2005: 96 ff., 140 f., 231; 
2006). Against the background of these findings, it can be hypothesized that 
the extent to which students benefit from a contrastive approach increases 
with the level of translation competence they have acquired. The students 
under scrutiny in the present study were all translation novices, for whom 
a monolingual approach turned out to be the more effective one. From 
this finding, however, it cannot be concluded that this would also be the 
case for students with an advanced level of translation competence. There 
might be a break-even point with regard to translation competence at which 
a contrastive approach of teaching writing skills will start to yield better 
results than a monolingual one. A monolingual approach indeed seems to 
be the approach to be preferred for translation novices. Further research 
involving students of various translation competence levels will be needed 
in order to delve deeper into this topic. 
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Apart from writing course effectiveness, the present study has also shed 
some light on the development of writing competence and, more specifi- 
cally, on the development of individual writing sub-competencies, such as 
formal, lexical, grammatical, textlinguistic and compositional (stringency) 
competencies, as components of the dynamic system ‘writing competence’ 
in relation to each other. The development of writing competence and its 
sub-competencies becomes visible in Figures 5 to 8. In these four figures, the 
patterns at the right ends of the diagrams are more parallel (harmonious) 
than the patterns at the left ends of the diagrams. Both for the L1 (German) 
and for the L2 (English), the stretches of the x-axes which show this more 
parallel pattern are larger for the results obtained at the end of the courses 
than for the results obtained at the beginning. This suggests that a higher 
level of writing competence is reflected not only by higher overall scores 
but also by a more harmonious relationship between the scores in each of 
the five rubrics, which are assumed to reflect relevant sub-competencies of 
writing competence. This finding is in line with dynamic systems theory 
where the reaching of a higher developmental stage has been found to co- 
incide with more stable or harmonious patterns, whereas initial phases of 
new competence development bursts show in a large variability in the set 
of variables which form the competence system (Thelen/Smith 1994: 97). 

The striking differences in the patterns in Figures 5 to 8 suggest that it 
might be worthwhile to investigate whether the patterns that occur in the 
area charts can be correlated with competence levels such as the levels A1 
to C2 (at least with regard to the skill ‘writing’) in the Common European 
Framework of Reference for Languages (CEFR). A similar endeavour of cor- 
relating less advanced learner text characteristics with competence levels of the 
CEFR (maximum B1) has already been undertaken by Verspoor, Schmid & 
Xu (2012). The competence levels of the students who, in the study presented 
in this article, scored 13 or more points might reflect the level C1 or C2. Since 
students are required to have achieved level B2 of the CEFR in order to be 
admitted to a BA programme in English language and literature at Justus 
Liebig University, the average scores of the students who have English as a 
major or minor should reflect at least this level. The fact that only students 
who study English obtained more than 8 points might suggest that the scoring 
system used for the present analysis is sensitive enough to allow for a differ- 


entiation between at least the competence levels of B1 upwards with B1 being 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Writing competence profiles as an assessment grid? 131 


the level of English that all German students are expected to have mastered 
when leaving high school and B2 representing the entrance level required for 
an English programme. Figures 7 and 8 then also show, however, that not all 
students admitted to English programmes fulfil this requirement in practice. 

In line with McCutchen’s (1996) capacity theory of writing, the hypoth- 
esis can be formulated that a high stringency score in combination with a 
high score for textlinguistic correctness can only be obtained when the sub- 
competencies at the lower levels have been automatized to a certain extent. 
This hypothesis is based on the assumption that to achieve textlinguistic 
correctness and stringency, an author has to be able to consider an amount 
of co- and context that is rather complex compared with the amount of 
co- and context that has to be considered in order to achieve correctness in 
the rubrics which represent less complex text-production sub-competencies. 
Taking into consideration this larger context requires considerable working 
memory capacity. This working memory capacity can again be assumed 
to be available only for authors who have acquired a certain amount of 
confidence and routine with regard to the less complex sub-competencies. 
Whereas the findings for L1 writing corroborate this hypothesis, the scores 
that students obtained at the end of their L2 writing courses (see Figure 8) 
indicate that even students whose lower-level sub-competencies were still 
underdeveloped managed to obtain relatively high scores for textlinguistic 
correctness and stringency. An explanation that reconciles this finding with 
the capacity theory of writing is that macrostructures of argumentative 
texts as well as the development of a line of argumentation were focused 
on in the writing courses that students had attended. This may have led to 
routines for the organization of such texts which then allowed the students 
to compose stringent and coherent texts without the high cognitive load 
that this would involve when authors were confronted with the composi- 
tion of texts for which they did not have fixed genre patterns in their long- 
term memories to fall back on. If this explanation is correct, this will also 
mean, however, that the competences students have acquired specifically 
for writing argumentative texts may not be transferrable to composition 
tasks that involve different genres. Expert writers are flexible in adapting 
their higher-order sub-competencies, such as textlinguistic competency and 
stringency, to new genres, whereas students who have just acquired rou- 
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tine with regard to a very limited repertoire of textual patterns are not (cf. 
Steinhoff 2007: 330)*. 

The fact that the lexical and grammatical error ratios were found to cor- 
relate at both the beginning and the end of the English courses and at the 
beginning of the German writing courses corroborates the finding from DST 
applied to second-language acquisition research that lexicon and grammati- 
cal repertoire are connected growers (de Bot/Lowie/Verspoor 2007: 19). No 
explanation can be provided for the fact that their correlation at the end of 
the German writing courses was non-significant. 

The present study has also provided insight into the interrelationship 
between L1 and L2 writing skills. The finding that the error ratios at the 
lexical and grammatical levels were significantly higher in L2 composi- 
tion than in L1 composition corroborates earlier findings (see Section 2.2). 
Against the background of the capacity theory of writing, the higher cogni- 
tive load involved in decision making at the lexical and grammatical levels 
in the L2 is a possible explanation for the lower stringency values in the 
L2 as compared with the L1. 


6 Conclusion 


The methods of data collection and analysis presented in this article have 
allowed us to take stock of the linguistic areas in which students commit 
errors when composing texts, especially argumentative ones, in their L1 and 
their L2. From the findings presented, conclusions can be drawn as to the 
topics that should be covered in compensatory writing courses at university 
entrance level. We do not claim, however, that the topics mentioned in this 
article are exhaustive. 

The methods employed in the present study have also allowed us to 
‘measure’ the impact that one semester of writing instruction has on the 
development of students’ writing skills in their L1 and L2. The study has 
also allowed us to glimpse into the relationships between writing sub-com- 


4 With reference to Jechle (1992: 55), Steinhoff (2007: 330; our translation) states: 
“From a social-communicative perspective, highly developed writing skills can- 
not be assumed before writers are able to adapt to their readerships in a flexible 
manner, i.e., before they are able to appropriately address their readerships in 
various texts or genres.” 
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petencies and their development in relation to each other over time as well 
as into the interdependence between writing skills in students’ L1 and L2, 
where findings of earlier studies could be replicated. 

To gain a more complete picture of different stages of writing competence 
development, the corpus of writing data analysed for the present study 
needs to be extended to also include data of highly proficient writers on 
the one hand and writers with a lower competence level than the students 
under scrutiny in this article on the other hand. In order to investigate the 
influence of a contrastive or translation-oriented approach on the develop- 
ment of writing competence, the corpus should also comprise data from 
participants with different levels of translation competence. Ideally, data 
for both languages should be collected from the same writers composing 
in each of the two languages, whereas in the present study, the L1 and L2 
writers were not identical. 

A comparison of area graphs from populations with different levels of 
writing and translation competence could eventually allow us to address 
the question whether patterns in the area graphs can be correlated with 
competence levels, such as the ones of the CEFR. Such comparisons could 
then also give us more profound insights into the development of individual 
sub-components of writing competence in relation to each other over time. 
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Appendix A 
Assignment: 


Imagine you are writing for a student magazine that covers current con- 
troversial issues. For this magazine, compose a text (approx. 250 — max. 
350 words in length) about one of the following topics. Your text should 
be comprehensible (understandable) to your fellow students, i.e., those who 
are in the same semester at university as you are and have little background 
knowledge about the topic. 


1. Free music for everyone! The Pirate Party in Germany would like to make 
all music legally available for free download on the internet. Should the 
copyright restrictions on music and other forms of intellectual property 
be removed? 

2. In an attempt to cut down on pollution in big cities, Germany has created 
so-called “Umweltzonen” (green zones) in cities such as Berlin, Frankfurt 
and Hamburg, where the driving of automobiles which do not meet cer- 
tain environmental criteria is forbidden. Some even recommend banning 
the driving of cars altogether in big cities. Should there be a general ban 
on cars in big cities? 

3. After having been briefly introduced in 2007 and 2008, tuition fees for 
university students in Hesse have again been abolished. Should tuition 
fees be reinstated in the state of Hesse, or should universities charge no 
tuition to attend them? 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


138 S. Göpferich / I. Neumann 


Appendix B 


Example of an annotated text with minimal stringency (0 points) 


The utopia of getting to work in time 


Who does not know the decision <semantic: denotation> between taking the 
bus or <semantic: denotation> driving with the <specifier> own car? There 
are enough aspects <semantic: denotation> speaking <syntax> for each 
of it <case, number, agreement>. In Germany, experts want to make this 
decision easier for travelers: they simply want to ban driving with <preposi- 
tion> cars in big cities altogether. A brief overview of the advantages and 
disadvantages of prohibiting cars in German cities <syntax>. <formatting> 
7 am. <spelling> <punctuation> and the traffic jam near Frankfurt seems 
endless. 30 minutes left until work starts. The same every day <genre/idi- 
omaticity>: thousands of workers driving into the city every morning. The 
same number driving outside <semantic: denotation> every afternoon. The 
nerverending <spelling> traffic <collocation> is not just bad for the nerves of 
the poor drivers, it is even worse for the environment around <preposition> 
Frankfurt, Berlin, <syntax> Hamburg... <formatting> So why not taking 
<word form> the bus or <specifier> train? Especially with trains <punctua- 
tion> there is no risk of getting stuck in the rush hour. Big cities as the above 
<syntax> can be reached from nearly everywhere <punctuation> and the 
people who have to go there <text coherence> for work will <modality/illo- 
cution> not live too many kilometers away. Another plus: the infrastructure 
in Germany is highly developed and gets improved regularly. Waiting two 
hours for the next bus was a long time ago <idiomaticity/genre>. You pay for 
it <text coherence> and save lots of money you would <syntax> have <tense> 
to spend for parking lots <semantic: denotation> or tickets because, in a 
hurry, you passed <semantic: denotation> a red light. Avoiding traffic jams, 
<punctuation> help <word form> the environment and <punctuation> by 
<preposition> doing so <infinitive/participle>, save <repetition word form> 
a lot of money-<formatting> there aren’t any disadvantages of banning all 
cars <implicitness>, are there? <syntax> Of course this <text coherence> can 
<modality/illocution> just be an <specifier> utopia. <repetition formatting> 
Taking the bus or the train is a good thing to do <punctuation> but in times 
of decreasing <semantic: denotation> prices every year <syntax>, it became 
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<tense> a luxory <spelling> and impossible for people with low wage <word 
form>. Workers who cannot afford the 30 Euros for the train from Gießen 
to Frankfurt every day will be excluded <semantic: denotation> from public 
transport completely. Furthermore, the train or the bus leaves. <punctua- 
tion> With <semantic: denotation> you <syntax> in it or not. Your car 
is always waiting <aspect> for you. A great plus for all vehicles who will 
bring you to work late just because of your own fault. <sense> Flexibil- 
ity, independence and always on time <syntax> — advantages no German 
worker does not want to have. If there will not be <tense> a change in mind 
<semantic: denotation><formatting>- and in the price-policy-<repetition 
formatting> banning cars in big cities will stay what it is today: wishful 
thinking. (410 words) 


Example of an annotated text with maximum stringency (3 points) 


Warum das Autofahren in Großstädten generell untersagt werden 
sollte - ein Betrag <spelling> von [XY], ein umweltinteressierter 
Student <case, number, agreement> der JLU Gießen 


In Freiburg, Deutschlands selbstproklamierter <word form> „GreenCity“ 
<punctuation> hat man nun schon seit einigen Jahren Erfahrungen mit 
einem nahezu vollständigen Verkehrsverbot in der Innenstadt sammeln kön- 
nen. Statt auf den PKW greifen Freiburger nun wie selbstverständlich auf 
das sehr gute innerstädtische ÖPNV Netz <spelling> zurück, welches durch 
eine Wiedereinführung des Tram-Netzes enorm an Attraktivität gewonnen 
hat. Dies erlaubt ein stressfreieres Vorankommen in der Innenstadt und 
trägt zur großen Akzeptanz des Verbots in der Bevölkerung bei. Finer Um- 
frage des Freiburger Geographischen Instituts zufolge, <punctuation> sind 
<case, number, agreement> die Mehrheit der Bürger, <punctuation> sehr 
zufrieden mit dem Verbot. Den weiteren Gründen für die breite Akzeptanz 
des Verkehrsverbots möchte ich im Folgenden nachgehen und aufzeigen, 
warum das Freiburger Modell auch in anderen deutschen Städten umgesetzt 
werden sollte. 

Eine Auswertung der Umfrage der freiburger <spelling> Geographen 
ergab, dass ein wichtiger Grund für die breite Akzeptanz nicht etwa dem 
Ursprünglichen <spelling> Ziel, der Verbesserung der Atemluft, zugeschrie- 
ben werden muss, sondern dass Freiburgs Einzelhandel nun viel problem- 
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loser auch fußläufig erreicht werden kann. <punctuation> Ein positiver 
Umstand, der von allen Befragten hervorgehoben wurde. Nicht nur fiir den 
innerstädtischen Einzelhandel, auch für die Freiburger Gastronomie und 
Hotellerie hat sich die Abschaffung des innerstädtischen Verkehrs als wah- 
rer Geldsegen erwiesen. <formatting> Freiburg, die Stadt der 1000 Bächlein, 
liegt in der Nähe des Drei-Länderecks <spelling> und zog schon immer 
Wochenendtouristen, vor Allem <spelling> aus der Schweiz, an. Seitdem 
sich die Stadt, Dank <spelling> des Verkehrsverbots, als „Shopping-Metro- 
pole“ des Südwestens fest etabliert hat, verzeichnet Sie <spelling> nun auch 
unter der Woche Gäste aus allen Nachbarländern. Vor Allem <repetition 
spelling> die Franzosen schätzen die ruhige Innenstadt mit den zahlreichen 
Möglichkeiten zum Flanieren und Einkehren und fallen auch unter der 
Woche regelrecht in der schönen Zähringerstadt ein. Die Freiburger selbst 
bewerten das neue Interesse an ihrer Stadt im Ausland als sehr positiv und 
sind stolz auf ihre grüne Stadt mit dem neuen internationalen Flair. 

Das Beispiel Freiburg zeigt, <punctuation-> nach Meinung des Autors, 
<-punctuation> anschaulich, wie eine Regelung, welche ursprünglich der 
Verbesserung der Umwelt <collocation> dienen sollte, auch viele andere 
positive Effekte, vor Allem <repetition spelling> für den innerstädtischen 
Einzelhandel, haben kann. Insofern kann eine flächendeckende Einführung 
von Autoverboten in allen deutschen Innenstädten nur begrüßt und aktiv 
vorangetrieben werden. Eine wichtige Voraussetzung für das Gelingen und 
für die breite Akzeptanz dafür ist jedoch ein sehr gutes ÖPNV-Angebot in 
der Innenstadt und, damit einhergehend, die Schaffung von ausreichend 
Pendlerparkplätzen am Stadtrand. 

Um dies zu erreichen, sollten zukünftige autofreie Städte als Vorbereitende 
<spelling> Mafsnahme eine Wiedereinführung der Trambahnen überlegen 
<semantic: denotation>, welche wegen ihres sauberen und kosteneffizienten 
Elektroantriebs in Fachkreisen als idealer „grüner“ Autoersatz in Grofsstad- 
ten angesehen werden. Außerdem ist der Bau einer Tram-Infrastruktur kos- 
teneffizienter als der Neubau eines U-Bahnnetzes, der besseren Alternative 
für finanzkräftigere Großstädte. (443 words) 
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Portfolios as a means of developing 
and assessing writing skills! 


Abstract 

English: Writing portfolios can serve various purposes: They can be used both as 
a tool for supporting and facilitating learners’ individual learning processes and as 
an assessment tool, which rather focuses on learning outcomes and achievements. 
Through the use of portfolios, improved learning processes, a more holistic ap- 
proach to assessing writing skills, and, ultimately, better learning outcomes are 
expected. The aim of this study is to explore a teacher’s and her learners’ actual use 
of a portfolio as well as their perception of portfolio work in the context of foreign 
language writing instruction in order to reveal patterns and generate hypotheses 
on portfolio work. Therefore, this study has a strong focus on the actual use of 
the writing portfolio and the circumstances influencing portfolio work. The major 
hypothesis generated in this study through qualitative research methods relates to 
a gain-loss effect of portfolio use. It suggests that the introduction of portfolios in 
the writing classroom cannot just be viewed as an additional offer to the students 
and their learning but can also necessitate abandoning established elements and 
procedures. 


German: Schreibportfolios erfiillen verschiedene Funktionen: Sie eignen sich einer- 
seits dazu, den Schreibprozess zu unterstützen und die Individualität der Lernenden 
zu fördern. Andererseits können sie als Instrument der Leistungsfeststellung genutzt 
werden, wobei weniger der Prozess als vielmehr die Lernergebnisse im Vordergrund 
stehen. Durch den Einsatz von Portfolios werden eine Unterstützung von Lernpro- 
zessen, die Möglichkeit eines holistischen Zugangs zur Feststellung von Schreibfer- 
tigkeit sowie letztendlich bessere Leistungen erwartet. Ziel dieser Studie ist es, den 
Umgang einer Lehrerin und einer Gruppe von Studierenden mit dem Portfolio sowie 
ihre Wahrnehmungen hierzu im Kontext der fremdsprachlichen Schreibförderung zu 
ergründen, um mit der Aufdeckung von Mustern Portfolioarbeit besser zu verstehen. 


1 I would like to thank Clive Earls, Martha Gibson, Claudia Riemer, Helen Kelly- 
Holmes, Antoinette McNamara, Lorraine Barron and the anonymous reviewers 
for their helpful comments on an earlier version of this article. 
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Dabei liegt das Hauptaugenmerk auf der tatsachlichen Verwendung des Portfolios 
und dessen Wahrnehmung. Die zentrale Hypothese, die im Rahmen dieser Studie als 
Ergebnis qualitativer Forschung aufgestellt wird, beschreibt einen Gewinn-Verlust- 
Effekt der Portfolioarbeit und nahrt die Annahme, dass Portfolioarbeit nicht nur als 
Gewinn verstanden werden, sondern in verschiedenen Bereichen auch einen Verzicht 
auf zuvor genutzte Elemente und Strukturen bedeuten kann. 


1 Introduction 


Learning portfolios are widespread in education. They are used in early 
childhood education, in all types of schools and educational settings, as 
well as in teacher training. In the foreign language classroom, the European 
Language Portfolio is prevalent but open portfolio formats are also used, 
for example audio portfolios focusing on listening and speaking skills, and 
writing portfolios. The latter are the focus of this study. 

The use of portfolios is expected to improve teaching and learning on 
several levels, for instance by individualizing learning, promoting learner 
autonomy and by offering an opportunity for a fair and holistic assess- 
ment of complex skills or competencies (Grittner 2009; Kara 2007). Re- 
cent studies optimistically show that portfolio use in the foreign language 
(L2) writing classroom can reduce writing anxiety (Öztürk/Cecen 2007), 
help to improve self-assessment skills (Poppi/Radighieri 2009) and lead 
to better texts (Aydin 2010; Khodadady/Khodabakhshzade 2012; Tezci/ 
Dikici 2006). These results show that the positive effects of portfolios can 
encompass improved language and writing skills, improvements with regard 
to metacognitive and strategic aspects as well as changes that concern the 
affective level. However, little is known about the process of portfolio work 
that leads to these outcomes. 

This article first focuses on the potential uses of portfolios and will then 
investigate a teacher’s and her students’ perceptions of portfolio work as 
well as the underlying assumptions that motivate their actions. The research 
questions are: How does the teacher use the portfolio to promote and assess 
writing skills? How do students perceive the portfolio as a tool to improve 
their writing skills? How do they perceive portfolio-based assessment? 
Based on this deep understanding of the processes of portfolio use and the 
underlying assumptions and attitudes, implications for the development of 


tailored portfolio concepts will be presented in the last part of this article. 
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2 Learning portfolios 


In general terms, a learning portfolio is understood to be 


“a purposeful collection of student work that exhibits the student’s efforts, pro- 
gress and achievements in one or more areas. The collection must include student 
participation in selecting contents, the criteria for selection, the criteria for judging 
merit, and evidence of student self-reflection.” (Paulson et al. 1991: 60) 


Several features that are common to all kinds of portfolios are subsumed 
under this rather broad concept: Portfolios are tools to document learning 
for the learners themselves as well as for others, including teachers, class- 
mates and parents. Moreover, they are tools of reflection and assessment. 

As portfolios focus on learning products as well as on the learning pro- 
cess, they may include proof of the learner’s accomplishments as well as of 
poor or even failed attempts. This understanding of learning and achieve- 
ment differs crucially from widespread forms of ‘red-ink’ writing instruction 
that focuses on correctness. 

Having their origin in the progressive and reform pedagogy of the late 
19th and early 20th centuries, learning portfolios emanated from a de- 
sire to renew educational policy with an emphasis on facilitating the in- 
dividual’s learning process and personal development. This included an 
understanding of learning as a meaningful, complex and individual process 
(Hacker 2008: 27). These ideas were introduced into the writing classroom 
in the 1950s with writing folders in Great Britain (Hamp-Lyons/Condon 
2000: 15) and in the 1970s with writing portfolios in the United States 
(Calfee/Freedman 1996: 4), where the concept was refined and further 
developed. 

The current concept of portfolio does not just refer to the actual folders 
but is rather the manifestation of a specific approach to teaching with a set 
of assumptions, decisions and techniques. This allows, and at the same time 
requires, the adaptation of this approach to the specific classroom context 
with the given conditions, especially with learners’ individual prerequisites 
and needs (Ballweg/Brauer 2011: 4). 


3 Portfolios in the writing classroom 


The term writing portfolio refers to a form of portfolios which centres on 


writing skills, even though reading skills and all other aspects of language 
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use are also included. They are expected to make working and writing pro- 
cesses as well as the individual writer’s skills more visible and, in doing so, 
are expected to raise the students’ awareness of their own writing processes 
and support them in improving their writing skills. 


3.1 Documenting and facilitating the development of writing 


skills 


The broad understanding of the term writing skills, as it is also used in 
the context of portfolio work, relates to more than just the ability to write 
correct and well-structured texts. Writing skills are not solely measured by 
the output, in this case a good text, but also by an adequate process. For 
writing in a first language (L1), Fix (2006: 23) acknowledges four subsets 
of skills which constitute writing skills: 


e knowledge-based skills which include knowledge of the topic but also 
of text types and language use 

e methodological skills which enable writers to organize their writing process 

e social skills which allow writers to anticipate their readers and to write 
reader-based texts 

e personal skills which include the writers’ capacity for self-regulation 


For L2 writing, language skills, knowledge of culturally specific features of 
text types and intercultural competence need to be added to the list. Yet, to 
be a skilled writer does not only mean to possess these skills but also that 
they can be translated into action (Beaufort/Inesta 2014: 149-151). Based 
on this understanding, writing skills need to be developed by completing 
writing tasks which have to be specific, embedded in a context and com- 
municative (Ballweg 2015a: 102). 

Portfolios are not only a place to display the students’ own writing skills 
but in addition to this, they offer numerous opportunities to practice writ- 
ing, as they include a large amount of written documentation and reflec- 
tions (Glaser-Zikuda/Lindacher 2007: 192). In this manner, the students’ 
writing skills are expected to be promoted. In the L2 classroom, this type 
of ungraded writing assignment is found to be helpful as the students’ texts 
tend to get longer and syntactically more complex with practice (Bonzo 
2005: 124; Delett et al. 2001: 559 f.). 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Portfolios as a means of developing and assessing writing skills 147 


3.2 Representation of the writing process 


Portfolios were introduced into the writing classroom when the focus of 
instruction and assessment shifted from the product of writing to the writ- 
ing process. They display the writing process as they include first drafts of 
texts, comments and feedback by peers and teachers, learning logs, reflec- 
tions on the text as well as the comments, revised drafts and a final version 
(Brauer 1997: 5). This allows readers to reconstruct the individual learner’s 
writing process. 

The underlying understanding of the writing process is often explained 
with reference to Hayes & Flower’s model of writing with its three iterative 
and recursive phases of planning, translating (i.e., formulating) and review- 
ing (Hayes/Flower 1980: 11). Despite the criticism levelled against Hayes & 
Flower’s early writing model that it oversimplifies writing processes (e.g., 
Molitor-Lübbert 1996: 1006-1008), this model serves quite well, and in 
fact, better than Hayes’ later models (1996; 2012), to explain the actual 
steps of students’ use of writing portfolios, as notes, first drafts and revisions 
are collected and arranged to illustrate the individual’s working process. 

Yet, the above-mentioned criticism, which is also closely related to post- 
process theory, has to be taken seriously. The model of a writing process 
of planning, writing a first draft and revising does not reflect the writing 
process of all writers and, therefore, can be considered as over-generalizing 
(Carstens 2008: 83 f., 93). From a post-process theorist’s point of view, 
successful writing instruction should be more flexible and individual and 
should include creating manifold learning opportunities, offering oppor- 
tunities for interaction and the negotiation of meaning and facilitating the 
development of language awareness (Carstens 2008: 86-92), stressing many 
features portfolio-based instruction includes with its profoundly individual, 
situational and context-bound view on writing (Ball/Ellis 2008: 504-510). 
This approach, however, suggests applying the three steps of planning, for- 
mulating and revising less strictly as there might be, for example, students 
who do not work with first drafts in a narrow sense. 

The role of feedback and interaction with peers in the writing process 
and in writing portfolios as mentioned above can be illustrated using a 
writing process model by Becker-Mrotzek & Böttcher (2006: 27). The au- 
thors emphasize writing as an asynchronous way to communicate and refer 
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to Ehlich’s idea of a “zerdehnte Sprechhandlung” (extended speech act) 
(Ehlich 1983: 32). The anticipation of a readership with its knowledge, 
expectations, interests and authority strongly influences individuals in their 
writing process (Becker-Mrotzek/Böttcher 2006: 27). 

While these aspects of writing processes and feedback apply to first- and 
second-language writing, there is a specific feature to L2 writing portfolios: 
As the first language is considered a crucial part of second-language writ- 
ing processes (Börner 1989; Grießhaber 2006; Krings 1989), it might be 
assumed that L2 writing portfolios are bi- or multilingual (Ballweg 2015a: 
109). However, this is neither the case in any of the research quoted here 
nor in the writing class that formed the context of my own study. In general, 
in many cases L2 writing portfolios do not represent L1 aspects of the L2 
writing process. 


3.3 Feedback and social interaction 


It is self-evident that feedback and interaction with potential readers is es- 
sential for developing the ability to write reader-based texts: 
“The consideration of written language as communicative tool implies that every 
text has an audience. From this point of view, the idea of negotiation between 


writer and reader is introduced into the composition process.” (Milian Gubern 
1996: 273) 


This kind of negotiation between writer and reader can immediately help 
writers to improve the texts on which they are working. Furthermore, it can 
promote their general writing skills, especially with regard to their ability 
to anticipate the reader’s perspective. In her meta-analysis of research on 
writing, Porto (2001: 40) concludes that writing exercises have only little 
effect on writing skills if no feedback is provided. 

The most common form of feedback is comments by teachers. The find- 
ings on effective teacher feedback, however, are contradictory and incon- 
clusive, as Busse (2015) shows in her meta-analysis of recent research on 
the influence of feedback on writing motivation. This is also true for other 
aspects: whereas O’Brien (2004) emphasizes that feedback on content and 
structure is most effective, Gascoigne shows that L2 learners can use feed- 
back on correct language use to revise their texts in most cases but do not or 
cannot react to feedback on content and structure (Gascoigne 2004: 74 f.). 
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These findings suggest that it can be difficult to measure the effectiveness of 
feedback due to the multiple and partly interdependent factors that have 
an impact on the writing process. 

To successfully integrate feedback in writing and learning processes, 
learners have to take the responsibility and demand feedback at the indi- 
vidually relevant stages in their writing process, which is something they 
also have to learn (Cresswell 2000). 

The need for training is also evident in the provision of peer feedback 
(Kamimura 2006; Rahimi 2013), which is considered a low-threshold form 
of support (Bruffee 1984: 637; Hu/Ren 2012: 69, 79). Peer feedback as a 
form of social interaction (Brauer 1996: 40) and as a means to mirror the 
recent status of a text (Brauer 2004: 24) requires learners to become read- 
ers of their peers’ texts and to ask questions which then serve to establish 
how a text can be understood and perceived by an interested reader (Bruffee 
1984: 641). 

Peer feedback differs from feedback given by teachers and can be under- 
stood as being complementary (O’Brien 2004: 9). If learners comprehend 
its different function, they show a high acceptance of peer feedback and are 
able to use it to improve their writing. If they feel that peer feedback is of- 
fered instead of their teacher’s feedback, they do not consider it useful, as a 
study with 116 learners of English in China shows (Hu/Ren 2012: 78). This 
may be one of the reasons for the negative attitude towards peer feedback in 
L2 learners (e.g., Kasanga 2006), which contrasts with findings from studies 
where it was seen rather positively (Ballweg 2015b; Kamimura 2006: 12; 
Mawlawi Diab 2010: 92). 

In general, peer feedback is considered a useful supplement to teacher 
feedback as it focuses on comprehensibility rather than on correctness. 
However, several factors can influence the success of peer feedback, such 
as the learner’s age, the number of learners in the class and their language 
proficiency (Rollinson 2005: 29). Learners need to be prepared to both give 
and receive feedback. Furthermore, an atmosphere of trust is essential to 
allow helpful feedback. It must then lead to action, which again requires a 
setting in which the revision of texts is valued (e.g., Keller 2010: 72). 
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3.4 Reflecting the individual writing process and achievements 


The awareness and metacognitive control of one’s own writing process is 
essential for successful writing (Glaser/Brunstein 2008: 371; Schoonen/ 
Glopper 1996: 99). This includes the monitoring of the individual writing 
process, the setting of aims, the evaluation of results at different stages of the 
process and the awareness of affective aspects (Glaser/Brunstein 2008: 374). 

Writing portfolios are used as a tool to reflect on strengths, weaknesses, 
personal objectives as well as those specified by the teacher, the individual 
writing process, and more. As it is assumed that the monitoring and control 
of the writing process does not automatically develop with writing experi- 
ence (e.g., Aziz 1995: 22), portfolio-based writing instruction also includes 
the adoption of new writing strategies. 

The reflection in the portfolio and in conferences with peers and teach- 
ers are shown to have positive effects on the writing performance. Khoda- 
dady & Khodabakhshzade (2012) focus on the students’ self-assessment 
activities and give an account of their significantly better results in a stand- 
ardized IELTS? writing test as compared to the control group. 

Portfolio-based writing instruction, and specifically reflective activities, 
are also shown to have positive effects on students’ ability to manage their 
own learning (Hung 2008: 139), their overcoming of writing anxiety (Kabi- 
lan et al. 2007) and on the quality of the texts they write. Aydin (2010: 481) 
presents the following findings from a study with 39 future teachers of 
English in Turkey: 

“[P]ortfolio keeping has beneficial effects on the improvement of vocabulary, 

grammar, reading and research skills, organization of paragraphs and composi- 

tions, punctuation and capitalization, giving and receiving feedback, paragraph 


and composition development methods and techniques, and qualifications of para- 
graphs and compositions” (Aydin 2010: 481). 


He shows that as syntax gets more complex, the students’ planning phase 
becomes longer and more elaborate, and the texts are more creative and 
original (Aydin 2010: 481-483). 


2 The International English Language Testing System (IELTS) is a standardized 
test for English as a Foreign Language. 
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What the above-mentioned studies have in common is that their au- 
thors strongly emphasize the potential of portfolio use and that their find- 
ings point in this direction as well. Despite this, some critical aspects are 
mentioned, for example, the students’ rejection of peer feedback (Aydin 
2010: 484). In general, more research into considering the settings and the 
way in which portfolios are used is required in order to better understand 
the described effects and the conditions that are necessary to achieve these 
effects. 


3.5 Portfolio-based assessment 


Another field in which portfolios are expected to have much potential is 
assessment. As language learning is considered a complex activity that re- 
quires knowledge, meta-knowledge and the capacity for self-directed learn- 
ing (Kohonen 1997: 11), the assessment of language learning should include 
all these aspects. Portfolio-based assessment seems to be an opportunity to 
transfer recent ideas of teaching, which are characterized, among others, 
by concepts of flexibility, diversity and equality (Conacher/Kelly-Holmes 
2007: 28), into assessment. In a modern understanding, learning achieve- 
ments are product- as well as process-oriented, individual, diverse, describ- 
able by the learners themselves and by others, communicable, reflectable 
and made cooperatively (Breuer 2009: 72-77; Legutke 2002: 107). This 
can neither be assessed using standardized tests that focus on vocabulary or 
morphosyntactic correctness, nor can it be assessed using essays. If learning 
takes place in complex contexts and in interaction with others, assessment 
cannot be detached from the context (Lissmann 2007: 276). 

Portfolio-based assessment can take account of most of these ideas. Prod- 
ucts and processes are both shown and contextualized. Achievements are 
presented over an extended period of time so that ideally development can 
be assessed (Hamp-Lyons/Condon 2000: 4). 

It must be considered that portfolio-based assessment may cause dif- 
ficulties: it is time-consuming and work-intensive for teachers and learners 
(Legutke 2002: 108) as well as difficult to integrate into a system with an 
institutional and curricular need for grades. While portfolios invite feedback 
on the individual learner’s development, they are not made for grading 
in a rather competitive learning environment, especially as grading might 
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hinder not only honest reflections (Calfee 2000: 285) but also the inclusion 
of poor first text drafts in the portfolio. The portfolio concept has to fit the 
respective learning culture or the learning culture has to develop in a way 
that allows for new approaches to assessment (Breuer 2009: 210-212). 

Even in a Montessori primary school where no grades are given, port- 
folio-based assessment is not without challenges, as Grittner (2009) points 
out in an extensive study on parents’, teachers’ and learners’ perspectives 
on portfolio-based assessment. In spite of the potential difficulties, she con- 
cludes that portfolios are useful to give students and teachers information 
on the learners’ achievements and developments. Most interestingly, the 
teachers’ feedback encompasses the students’ working processes but tends 
to neglect the learning content (Grittner 2009: 170). 

With regard to the effect of portfolio-based assessment in writing and 
drawing, Tezci & Dikici (2006) point out that the learners who were work- 
ing with portfolios produced significantly better texts than the control 
group. These positive consequences are chiefly attributed to the frequent 
feedback sessions with peers and teachers. 

Similar findings were made by Khodadady & Khodabakhshzade (2012) 
for 59 learners of English in Iran. The students who were working with a 
writing portfolio achieved better results in an IELTS essay test than those in 
the control group. For the examined aspects of writing performance, port- 
folio-based instruction and assessment seems to have clear effects. However, 
in these studies it is not always evident how portfolio-based instruction dif- 
fered from that in the control group and how the teachers’ motivation and 
commitment might have differed, and how this might have subsequently 
influenced the results. The broad understanding of learning outcomes dis- 
cussed above, and also described by these authors as a major benefit of 
portfolio use (i.e., the student’s individual development, their individual 
writing process, self-assessment, and reflection), is not represented in the 
studies, as eventually only text quality rather than personal development, 
the individual process and adjacent factors is measured. It can be assumed 
that the idea of learning achievements and assessment described above is 
hard to introduce into the classroom and into research likewise. 
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4 The study 


As stated above, portfolio work is based on a set of assumptions and tech- 
niques of foreign language teaching. Only few of the studies cited in this 
article that show the positive effects of portfolio use in the L2 classroom 
describe how portfolios were actually used in the classroom and thus which 
aspect of the portfolio has ultimately led to these positive effects. Therefore, 
my study (Ballweg 2015a) aimed at exploring the use and the perception of 
writing portfolios in German as a Foreign Language at tertiary level. In this 
chapter, I will present one aspect of this bigger study and elaborate on it. 

The study is based on the assumption that there is a major gap in re- 
search on portfolios because apart from an action research study on the 
introduction of the European Language Portfolio by Kristmanson et al. 
(2011), a vast majority of research was conducted in courses with experts 
on portfolio work as teachers, mostly in the form of action research. How- 
ever, in practice only few teachers in Germany using writing portfolios have 
actually been prepared to use them. For that reason, I focused on how an 
experienced language teacher with no prior experience in portfolio work 
introduces and uses this tool. 

For this purpose, interviews with individual students and the teacher 
as well as videography of classroom activities and portfolio conferences 
were used to gain insight into processes, the underlying assumptions and 
the individuals’ perceptions of portfolio work. The results show how the 
teacher is constantly struggling to find an adequate way of using the port- 
folio while a multitude of options is available and difficult decisions have 
to be made (Ballweg 2015a: Chap. 8). Moreover, the data show in detail 
which prerequisites make portfolio work positive and useful to learners and 
to the teacher (Ballweg 2015a: Chap. 9). 

In the following, I will focus on the use of portfolios in the L2 writing 
classroom and on the students’ and the teacher’s perceptions. 


4.1 The context of the study 


The study was conducted in the context of a course on “Writing in an 
Academic Context” in German as a Foreign Language for 16 engineering 
students at a German university. The level of language proficiency was B1 
to B2 according to the Common European Framework of Reference for 
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Languages (CEFR), and the course took place weekly with three hours per 
week over nine weeks with an additional session for portfolio conferences. 

The teacher was an experienced teacher for German as a Foreign Lan- 
guage. She had heard and read about portfolios and worked with check- 
lists from the European Language Portfolio before. In the semester before 
the study was conducted, she had tried to introduce portfolio work into 
a course with the same title. Still, she considered herself a novice in using 
portfolios in the classroom. 

The students were asked to keep a portfolio with several drafts of texts. 
Apart from three shorter texts of only one paragraph, the students were 
asked to write a summary, a discussion essay, a further essay, a CV and 
a cover letter for a job application. At the beginning of the semester, the 
teacher had intended to give more writing assignments but as portfolio 
work proved to be time-consuming, she refrained from this. 

Classes were used to give input on how to write a good text and to 
prepare for the writing assignments. The texts were written as homework 
and discussed with the peers in the session that followed. Two of the texts 
had to be handed in via e-mail for feedback but the teacher encouraged the 
students to hand in more than two texts so that she could better support 
the students. However, only one student made use of this opportunity. The 
others sent in two texts or even fewer. 

The teacher’s focus clearly was the students’ reflection on their individual 
writing processes and their knowledge about writing and texts. She offered 
them information on the structure of texts as well as on strategies for plan- 
ning, writing and revising texts. She did so by discussing these aspects in 
class and working with sample texts from the students. Moreover, she used 
peer feedback to encourage students to reflect on text quality in general and 
especially on the quality of their texts by applying the criteria discussed to 
the texts of their peers and to their own texts. 

The students kept a portfolio folder — digital or in print — that included 
their own learning biography, the first drafts and the revisions of the five 
texts (summary, two essays, CV, cover letter) and a learning log. The teacher 
gave several suggestions on what to include in the learning log but did not 
give any guidelines. So in most cases they consisted of notes on the con- 
tent of each class and a list of vocabulary. Apart from these instructions 


on the content of the portfolios, the students were free to decide on how 
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to organize their portfolios and what to include. The teacher encouraged 
them to include material from other language or engineering classes if they 
considered them useful to demonstrate their writing skills in that semester. 
For this reason, the portfolios that the students submitted for grading at 
the end of the semester differed significantly in length. 

In addition to the weekly classes, the teacher held a portfolio conference 
with groups of two to four students at the weekend before the last session 
in class. While portfolio conferences can have a wide range of foci, for 
this specific conference, the teacher asked the students to prepare general 
questions on the organization of their portfolios and to present their most 
recent versions of them. In most cases, the students took turns reporting 
on their work with the teacher giving feedback to each of them while the 
others were listening. Only in one group, the students started discussing 
their portfolios and their questions with each other. 


4.2 Research questions 


Whereas previous research on writing portfolios placed a strong empha- 
sis on the effect of portfolio use on writing proficiency (e.g., Khodadady/ 
Khodabakhshzade 2012; Tezci/Dikici 2010), this study aims to shed light 
on the usefulness of portfolios from the students’ and teacher’s perspective. 
More specifically, the following questions will be addressed: 


1. How does the teacher use the portfolio to promote and assess writing 
skills? 
2. How do students perceive the portfolio as a tool to improve their writing 


skills? 


3. How is the portfolio used and perceived as an assessment tool? 


4.3 Research methods 


As the research questions indicate, the study aims at an in-depth explora- 
tion of portfolio-based writing instruction over the course of nine weeks. 
Portfolio work is supposed to be understood from the perspective of teacher 
and students, considering that their views and perceptions are influenced by 
numerous individual factors and subject to change in a constantly chang- 
ing environment (Gibbs 2007: 5). Therefore, this study does not intend to 
develop a universally applicable explanation of portfolio work but rather 
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attempts to gain a deep understanding and to provide a rich description 
(Geertz 1973; Ponterotto 2006) of perspectives, influences and processes 
in a specific course at a specific time that — through a purposeful selection 
of participants, an extensive corpus of data and a traceable as well as com- 
prehensible research method - allows us to understand the use of portfolios 
in other contexts and forms a basis for further research. 

To this end, a qualitative research approach was used, which is character- 
ized by the aim of exploring and understanding a phenomenon with a focus 
on the individual, a holistic perspective and a consideration of the context 
(Mayring 2003: 20; Riemer 2014: 21; Riemer 2006; Settinieri 2012: 250 f.). 
Grounded Theory methodology (Glaser/Strauss 1967; Strauss/Corbin 1996) 
proved to be an adequate means for exploring an aspect of portfolio work 
that has not been investigated extensively before and therefore required a 
“systematic, inductive, and comparative approach for conducting inquiry 
for the purpose of constructing theory” (Bryant/Charmaz 2010: 1). The 
result is not the presentation of causal connections but rather the identifica- 
tion of patterns in the data that allow an understanding of a phenomenon 
on a more abstract level. 

For the generation of data’, the lessons and the portfolio conferences 
were audio-taped. The core of the 25 hours of audio-taped and transcribed 
data, however, was formed by four interviews with the teacher and three 
interviews with each of the seven students participating, which were con- 
ducted over the course of one semester. The students came from France (2), 
Australia (1), China (1), Brazil (1), India (1) and Iran (1) and were in their 
early 20s. Four of them were enrolled in degree programmes in engineering, 
three were exchange students in the same field. Four of them were female, 
three were male. The time they had already spent in Germany ranged from 
one month to two years at the start of the semester. 

After transcription, the data were coded in accordance with Grounded 
Theory.* The first step was the open coding of the data from the classroom 


3 From a constructivist perspective, I prefer the term data generation to data col- 
lection and data interpretation to data analysis as these terms better stress the 
subjectivity of perception and understanding. 

4 For an extensive discussion of Grounded Theory in language research, see Ball- 
weg (2015a: Chap. 6). 
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observations and from the interviews for which initially codes close to the 
data, or in-vivo codes, were used, which means that the wording used by 
an informant was used in a code itself or that the codes were very close to 
the literal meaning with a minimum of interpretation and abstraction. This 
step was followed by focused coding. 
“Focused coding means using the most significant and/or frequent earlier codes to 
sift through large amounts of data. Focused coding requires decisions about which 


initial codes make the most analytic sense to categorize your data incisively and 
completely” (Charmaz 2006: 57). 


Focused coding is used to bundle codes into categories. Hints to what aspect 
might be relevant are, among others, 


a) statements that are connected or contradict other research 

b) statements of general interest 

c) repetitions, breaks and changes in the narratives and emphasized ele- 
ments and aspects connected to former research 


The second step of data interpretation in accordance with Grounded Theory 
is axial coding. Here, the data are taken to a further level of abstraction. 
Data that have been segmented during the phase of the open coding are 
put together again. In the case of the study at hand, data from the portfolio 
conferences were added. According to Strauss & Corbin (1996: 118-131), 
the categories are arranged according to a coding frame that names a core 
category, which is a central phenomenon, and groups other categories 
around it to explain causes, intervening conditions, context, the stakehold- 
ers’ strategies, context and consequences (Strauss/Corbin 1996; Strübing 
2008: 28). In this manner, all facets of a phenomenon as well as its condi- 
tions and influences can be reconstructed. 

The third and final step is selective coding (Strauss/Corbin 1996: 94), 
where all the data are viewed again and deductively coded to support or 
dismiss the results of the axial coding. 

The results can, in the sense of Grounded Theory, lead to what is to be 
understood as alterable, flexible “middle range theory” (Merton 1968) that 
explains a phenomenon at a certain moment in a defined context and serves 
as ground for further research (Strübing 2008: 10). 
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5 Results: The students’ and teacher’s perspectives on 
writing portfolios 


The data in this study were coded and interpreted as described above to 
answer the research questions and to shed some light on the way the teacher 
used the portfolio in the writing classroom (5.1), on the students’ perception 
of the portfolio as a tool to improve their writing skills (5.2) and on the 
portfolio as an assessment tool (5.3). The findings can best be summarized 
by the term of a gain-loss effect (5.4). 


5.1 The teacher’s use of the portfolio 


To understand the students’ perceptions of the portfolio, the first step of 
data interpretation must focus on how the teacher used the portfolio. The 
aim is not to evaluate or even judge the teacher’s actions but rather to 
describe and retrace them. As there was only one teacher in this study, the 
results are not to be understood as saturated but they rather describe an 
individual teacher’s actions, perceptions and assumptions and thus form 
the basis for the interpretation of further data concerning the students’ 
perspectives. However, it can be anticipated that the results of this case 
study are of a high explanatory value and can be used as a starting point 
for further research. 

The teacher explained that she aimed to prepare students with a profi- 
ciency level in German of B1 to B2 according to the CEFR for academic 
writing, teaching them the features of different text types and helping them 
to understand their own writing process (UB 1, 28-48)°. In the second in- 
terview in the middle of the semester, she stated that through teaching this 
class, she would gain experience that would be necessary to use portfolios 
reasonably in the following semesters.° From this statement it becomes 


5 Explanation of abbreviations: UB (= Unterrichtsbesuch) — observed class; I_LP 
(= Interview Lehrperson) - interview with the teacher [plus number of interview 
I-IV]; I_[+ student’s name] [+ number] — interview with student [name] plus 
[number of the interview]. 

6 All data excerpts have been paraphrased or translated from the original German 
into English in line with the language policy of the volume. As translations are 
to be understood as a form of interpretation, the character of the original data 
excerpts may hence not be represented in its entirety. 
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evident that she was aware that her use of the portfolio showed problematic 
elements. She considered herself a novice in using portfolios and felt she 
was struggling to introduce the tool into her teaching, which showed in 
different ways in the data. Several statements like the one referred to above 
imply that, from the teacher’s point of view, learning how to use a portfolio 
is primarily based on trial and error, which corresponds with findings by 
Kristmanson et al. (2011). 

How a novice perceives portfolio work is closely related to the category 
of insecurity, which served as one of the head categories under which several 
other categories were subsumed during axial coding. In this manner, the 
quality and context of the insecurity could be described in more detail: Ele- 
ments of insecurity, doubts and fears showed and were explicitly mentioned 
at several points in all of the interviews with the teacher so that it can be 
understood that her insecurity was not an initial one that was caused by 
the confrontation with a new challenge but was present all the time. As the 
mentioning of fear and insecurity was characterized by a certain manner 
of speaking, namely stammering, frequent pausing, abrupt break-offs and 
the repeated use of modal particles with a mitigating function, especially 
a bit (“so=n bisschen”), these features served to identify other potentially 
relevant statements in the data. It is interesting to note that the teacher’s 
insecurity was related to portfolio activities and, in the data, did not show 
with regard to other aspects of language teaching and classroom manage- 
ment. A crucial element was the connection between her insecurity and 
her desire to promote the students’ individuality. When she mentioned her 
strong fear to give too much information about portfolios to the students 
and, in so doing, limit their individuality in learning (I_LP IV, 652-654), 
she made a connection that might not have been obvious to outsiders. 
However, the data showed that her understanding of supporting learners’ 
individual needs was more or less to leave them maximum freedom in all 
decisions and actions. For this reason, she introduced the portfolio with all 
possible functions at once, namely as a tool for reflection, self-assessment, 
personal development, learning, documentation, consultation and promo- 
tion of writing skills. Furthermore, she explained that it could replace a 
textbook, would help her to give the students feedback on their texts and 
that it could be used for assessment (Ballweg 2015a: 248). She left it to the 


students to shape the portfolio concept according to their needs, which they 
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actually did (Ballweg 2015a: Chap. 9). She stated that she did not want 
to give many explanations at the beginning but was prepared to be more 
specific when the students were seriously struggling with their portfolios 
(LLP I, 112-116). 

What might be interpreted as the teacher’s helplessness in using the port- 
folio, rather - or at least also - proved to be the effect of a misunderstanding 
of the concepts of learner autonomy and individualized language teach- 
ing. Her actions were guided by the general idea that portfolios, learner 
autonomy and individualized learning are positive but she did not have 
more profound knowledge on how to adapt these broad concepts to her 
teaching. Expecting students to become autonomous, to reflect their learn- 
ing and writing processes and to adapt the portfolio to their needs may 
have been too ambitious an endeavour on both sides that most likely was 
to be disappointed. 

The portfolio was used for several purposes and, with this multitude of 
options, was eventually reduced to its very basic function, namely arranging 
the texts. Helping students to arrange the portfolio also became the major 
focus of the portfolio conference and of the last classes. 

The students’ main aim in this class, however, was not to become more 
autonomous and reflected but rather to improve their writing skills and 
their general language skills (e.g., I Qian I/II, 104-138). Encouraged by 
what she had found out about the opportunities writing portfolios can offer, 
the teacher placed a strong emphasis on cognitive and metacognitive aspects 
of writing and aimed at teaching knowledge about texts, for example, about 
the structure of texts and the features of different text types. This focus 
made students aware of their writing process and induced them to reflect 
on writing strategies and text quality and to use peer feedback to stress the 
writer-reader relationship. All this led to less time left for writing, which was 
further reduced because the explanation and organization of the portfolios 
as well as activities of reflection and self-assessment proved to be rather 
time-consuming as well (I_LP II, 335-341).’ Other opportunities for writ- 
ing in the portfolio (learning log, reflection) were not extensively used, as 
most students used short bullet-point lists instead of fully formulated texts. 


7 This phenomenon also shows in Grittner’s study of portfolios in a Montessori 
primary school (Grittner 2009: 181). 
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The cut-back on writing activities not only showed in class but also in 
the teacher’s assessment at the end of the semester (I_LP IV, 234-239), 
when reflection and the organization of the portfolio became increasingly 
important, while writing skills seemed to matter less (see Section 5.3). 

In general, portfolios in L2 writing instruction offer a multitude of op- 
portunities that cannot all be used at the same time. Therefore, teachers 
have to make many decisions as to both the focus of their teaching of writ- 
ing and the use of the portfolio. The necessity to make decisions can reveal 
insecurities, incongruities and a lack of knowledge on the part of teach- 
ers that had previously existed but were only revealed through portfolio 
work. Instead of making decisions based on instinct, teachers have to set 
priorities, to consider the multiple aspects of writing and portfolio use as 
illustrated in Fig. 1. 


multiple aspects of writing instruction 


ze writing as social knowledge Ben 
8P interaction abouttexts uae 


need for priorities 
and focus 


multiple options of portfolio use 


documentation reflection self-evaluation assessment 


Fig. 1: Need for priorities and focus in using a writing portfolio 
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5.2 The learners’ perception of the writing portfolio 


As the students had initially expected a stronger focus on writing in this 
class, some of them were disappointed and did not find the class and the 
portfolio useful. This is of course in part due to the teacher’s specific use of 
the portfolio. Instead of perceiving the portfolio as a tool to promote their 
writing skills, these students experienced it as separate from their learning. 
Tom stated that rather than doing portfolio work he would have preferred 
to “learn German” (I_Tom I, 184-185). 

Other students were more interested in, or at least open to, learning 
more about text structures and specific features of text types. This was the 
case when they acknowledged that they would benefit from the emphasis 
on metacognitive learning (e.g., I_Nilesh I, 49-53; I_Aline II, 8-17). It is 
interesting to note that the teacher’s lengthy and repeated explanations on 
these benefits hardly changed the students’ attitudes: Those students who 
were open to this approach right from the start were more positive. Those 
who did not see any usefulness in it, especially Tom and Atena, rejected 
the portfolio for at least the first five weeks despite the teacher’s attempt to 
point out its benefits. Later, a change towards a more positive stance was 
observed in some students: They acknowledged that the display of their 
achievements in the portfolio made them feel better about their writing 
skills but they still had the feeling that the portfolio did not help them to 
learn German or to improve their writing skills. However, the affective di- 
mension (feeling proud) and the metacognitive dimension (having a broad 
repertoire of writing strategies available) were valued more highly than 
in the beginning (I_Atena III, 134-140; I_Qian M, 131-135; I_Tom IH, 
46-47). 

In general, students’ expectations and attitudes at the beginning of a 
course seem to have a stronger impact on their perception of writing port- 
folios than the teacher’s explanations. Their own experiences might be more 
likely to lead to changes in their perception of the usefulness of portfolio 
work. 


5.3 Assessment: Individualizing vs. arbitrariness 


Portfolio-based assessment also caused a wide range of reactions among 
the students. Their interest in good grades was a vital criterion for how 
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they judged portfolio work. Whereas weaker students recognized that port- 
folio-based individualized assessment was to their advantage (I_Renato I, 
117-122), others were afraid of arbitrariness and unfair grades: 

“and then to corr/correct it but somehow I am under the impression that maybe 

the teacher doesn’t like my my structure the structure of my portfolio and then 

I will get poor grades but there will be someone who is weaker in writing than 


me but knows how to structure everything well and then . that’s a bit not fair” 
(I_Atena III, 231-237; cf. also I_Atena I, 90-97).° 


As a matter of fact, by considering additional aspects of the students’ 
achievements in assessments, such as their individual development and the 
quality of their reflections on their texts, the traditional criteria of writing 
assessment, especially the quality of texts, are devalued. The extent of this 
devaluation can vary and was taken to an extreme by the teacher in the 
present study, who did not mark her students punitively (I_LP IV, 267-276). 
She rather stated that she did not feel able to judge the quality of texts (I_LP 
IV, 301-313), even though she had been doing this for years in her classes. 
There are at least two possible interpretations of this phenomenon: Either 
her insecurity concerning the portfolio led to insecurity in assessment or 
else an already existing insecurity concerning the assessment of texts was 
revealed through the use of the portfolio and the necessity of revealing 
criteria for assessment. Unfortunately, the data do not allow concluding 
statements on this aspect. 

Another relevant result of the study refers to the affective dimension of 
portfolio-based assessment. With feelings of insecurity, pride, dislike of 
the portfolio and fear of unfair assessment as explained above, the affec- 
tive dimension proves to be important in the understanding of portfolio 
work. In assessment, there is an additional aspect to be considered: As has 
been stated previously, the new opportunities that portfolio-based assess- 
ment offers and the holistic view of learners mean that the focus of assess- 
ment shifts from the learning outcome to the learner (see Section 3). In 
this specific class, personal reflections and a learning log were included in 
the portfolio. Moreover, the teacher asked the students to add a learner’s 


8 Explanation of transcription conventions: . — pause of 1 second; .. — pause of 
2 seconds; ... — pause of 3 seconds; = — amalgamation of words; “ — strong 
emphasis; / — break-up; (It) - loud; * — end of commented speech; # — laughter. 
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biography (UB 1, 246-248), which made the portfolio a very personal 
document. The example of one student, Atena, shows how this can lead to 
feelings of vulnerability: 

“erm ... well I don’t know . well . for the first time I under”stand why ms. x [the 
teacher] erm . she used to say don’t you feel good when someone criticizes it [the 
portfolio] and so on and I didn’t bother and it was okay with me (It) but now it is 
really my “product * # now if it get poor grades it might be - yes: - it won’t feel 
good” (I_Atena II, 281-289). 


The teacher had similar doubts when she stated that she was trying to be 
fair but was finding it difficult because due to the personal nature of the 
portfolios, she rather had the feeling she was judging a person instead of 
assessing their achievements (I_LP IV, 212-222). 

It seems that the personalization and individualization of writing classes 
through the use of portfolios collides with traditional ideas of assessment 
in the institutional set-up at a university. 


5.4 The gain-loss effect of portfolio use 


The phenomena described above may be summarized as the gain-loss ef- 
fect of portfolio use: While portfolios are expected to add something to 
the teaching and assessment of L2 writing - reflection, language (learning) 
awareness, individualization, peer feedback, intensified interaction with 
teachers, more attention to affective aspects, and much more - learners in 
this study also experienced portfolio use as subtractive. They stressed that 
peer feedback was introduced at the expense of corrections by the teacher, 
work on language was replaced by reflection, writing activities were cut 
back for organizational matters, and considering additional aspects in a 
more holistic approach to assessment led to a devaluation of the importance 
of text quality. 

Even though this effect is strongly related to the teacher’s use of the 
portfolio in this study, it has to be acknowledged that, generally, the addi- 
tion of elements to teaching and assessment in most cases inevitably leads 
to the subtraction of others. To concede that this is also true for the use of 
writing portfolios is an important basis for the development of suitable, 
context-specific portfolio concepts. 
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6 Discussion and conclusion 


The results of this study highlight the gain-loss effect of portfolio use but 
should not be reduced to it. The results explain how the complexity of the 
portfolio idea forces teachers to make choices on what to add to teaching 
and what to exclude and how learners perceive these choices as both gains 
and losses. 

The teacher in this study had to make numerous difficult decisions as 
she was confronted with the complex task of including writing portfolios 
in her teaching, which entails challenges as well as opportunities. She tried 
to introduce all possible functions at once in order to achieve a maximum 
benefit for the learners. This again was challenging and time-consuming for 
her and for her learners, so much so that the learners were under the impres- 
sion that they had to give up many aspects of learning that were important 
to them, and they did not appreciate the additional opportunities offered 
by the use of a writing portfolio. The teacher’s explanations did not suffice 
to convince them of the benefits. If they could be convinced, this was rather 
due to the actual experience of using the portfolio. 

One way of dealing with this dilemma is to consider the genuine benefit 
of introducing portfolio-related activities in relation to the loss that this 
involves in other areas. It is not a matter of introducing a portfolio with 
all aspects or not using it at all. Rather, teachers and decision-makers have 
to analyse the learning needs of each group of learners and the benefit 
each new element could bring. A matrix for analysing the prerequisites of 
portfolio use can be based on the model of positive influences of portfolio 
use (Ballweg 201 5a: 312), which comprises criteria in four different fields: 


e learning aims and expectations 

e the learners’ prior knowledge and personal features, such as their attitude 
towards mistakes and their self-efficacy 

e personal habits and preferred ways of learning and studying 

e institutionally and culturally influenced learning habits 


Based on an analysis of the prerequisites and aims of portfolio work, teach- 
ers have to develop an individual approach to portfolio work for each group 
of learners that encompasses the following considerations: 
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e The benefits of introducing writing portfolios in the L2 classroom should 
outweigh the necessary efforts and subsequent cut-backs on important 
contents and aims that this introduction might bring for the whole group 
as well as for individual learners. 

e The writing portfolio should not be an add-on to writing instruction but 
should be combined with the content to be acquired in such a way that 
it is clearly a tool to facilitate both the learning of writing and of content 
instead of an additional burden. 

e The focus of the writing portfolio should be clear and comply with the 
learning aims set by the institution and by curricula as well as by the 
learners themselves. 


The development of a suitable portfolio concept is extremely challenging 
for teachers as they have to be able to analyse all possible influences and 
they have to know all options of portfolio use and writing instruction well 
enough to make informed decisions. To be able to do so, it is necessary 
to prepare teachers for portfolio use and to support them in the process. 
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“Prepare an outline first and then just write 
spontaneously” — An analysis of students’ 
writing strategies and their attitudes 
towards professional writing 


Abstract 

English: Students’ beliefs about writing and writing strategies may interfere with 
their acceptance of writing support. Therefore, it is helpful for writing instructors to 
be aware of these beliefs and attitudes. This article provides insights into students’ 
attitudes towards planning and spontaneous writing, both for short assignments in 
non-academic genres and longer texts in academic writing. For this purpose, state- 
ments from 163 students were analysed, which were posted during four courses of 
the BA programme “Transcultural Communication” at the Centre for Translation 
Studies of the University of Vienna. The courses were designed around writing in 
German as an L1 or L2. The results reveal numerous individual writing strategies 
and beliefs about writing. Students who take the ‘conscious craft’ position are keen 
to learn more about writing strategies, whereas students with the ‘kiss of the muse’ 
position doubt whether writing support might be helpful for them at all. Between 
these opposing poles, various forms of individual habits of writing and beliefs about 
writing (strategies) occur. Additionally, the didactic potential of online forum discus- 
sions about writing was analysed. These written discussions provide first insights 
into students’ conceptions about writing and can help them to re-consider their 
beliefs and attitudes through exchange with others. Discussions can then be further 
expanded in subsequent lessons to resolve misunderstandings, to foster students’ 
reflection of their individual writing approaches and to design and administer as- 
signments geared to their specific needs. 


German: Was Studierende über Schreiben und Schreibstrategien denken, kann ihre 
Aufgeschlossenheit gegenüber Schreibdidaktik und Schreibbegleitung beeinflussen. 
Dementsprechend wichtig ist es fiir Lehrende, die subjektiven Schreibtheorien Stud- 
ierender und ihre darauf beruhenden Einstellungen zu kennen. Im vorliegenden 
Beitrag wird analysiert, wie sich Studierende selbst über ihr Schreiben äußern. Im 
Mittelpunkt des Interesses steht dabei das Spannungsfeld von Planen und Drau- 
flosschreiben, einerseits im Kontext wissenschaftlichen Schreibens, andererseits bei 
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der Produktion von kurzen Gebrauchstexten. Für den Beitrag wurden Statements 
von 163 Studierenden analysiert, die diese im Rahmen von vier Lehrveranstaltun- 
gen des Sprachbereichs Deutsch im Bachelor-Studiengang ,,Transkulturelle Kom- 
munikation“ am Zentrum fiir Translationswissenschaft der Universitat Wien auf 
Lernplattformen gepostet haben. Fiir einen Teil der Studierenden ist Deutsch die L2. 
Die Ergebnisse zeigen eine Vielfalt von individuellen Schreibstrategien und (ideolo- 
gischen) Annahmen über das Schreiben. Studierende, die Schreiben als „Handwerk“ 
betrachten, sind sehr interessiert an Empfehlungen aus der Schreibdidaktik und 
setzen sich gerne mit ihrem Schreibprozess auseinander, während Studierende, die 
einer „Musenkuss-Position“ anhängen, nicht sicher sind, ob Schreibdidaktik für 
sie überhaupt hilfreich sein kann. Zwischen diesen Extrempositionen finden sich 
vielfältige Formen individueller Annahmen über das Schreiben und individueller 
Schreibgewohnheiten. Weiterhin soll das didaktische Potential solcher Forendiskus- 
sionen über Schreiben und Schreibstrategien beleuchtet werden. Der Austausch mit 
Peers in den Online-Foren kann einen Ansatzpunkt dafür bieten, dass die Studier- 
enden ihre persönlichen Zugänge reflektieren. Die Diskussion kann - und sollte - im 
Präsenzunterricht fortgeführt werden, nicht zuletzt um Missverständnisse in Bezug 
auf das Schreiben und Zielsetzungen von Schreibdidaktik zu klären. Schreibdidak- 
tische Konzepte, wie sie im letzten Kapitel des Beitrags diskutiert werden, können 
die Reflexion über Schreibstrategien noch fortführen und erweitern. Ziel ist, die 
Studierenden bei ihrer individuellen Professionalisierung zu unterstützen. 


1 The context of the study: Writing instruction and its 
objectives at the Centre for Translation Studies of the 
University of Vienna 


Professional writing in a minimum of three working languages is considered 
a key competence in the Bachelor’s programme “Transcultural Communica- 
tion” at the Centre for Translation Studies of the University of Vienna.' The 
students are expected to acquire and develop the necessary language, text 
and cultural competences. Particular emphasis is placed on writing for spe- 
cific audiences and within intercultural contexts (Zentrum fiir Translation- 
swissenschaft 2011). In accordance with Beaufort & Ifesta (2014: 146 ff.), 
the learning objectives involve “subject matter knowledge”, “rhetorical 
knowledge”, “genre knowledge”, “discourse community knowledge” and 
“writing process knowledge”. 


1 For the entire curriculum, see Zentrum für Translationswissenschaft (2011). 
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At the Centre for Translation Studies, these objectives are addressed in 
several courses for text analysis and text production, which are offered at 
several levels and for a number of languages. While the courses are taught 
by different teachers, all of them address similar issues (albeit in different 
ways), and continuous exchange between instructors ensures comparabil- 
ity and a consistent quality of teaching. Most of the courses are taught in 
a blended-learning scenario. 

The tasks students complete in these courses constitute “exercises of 
rhetorical problem solving” of the sort described by Göpferich (2015: 181). 
The students have to compose relatively short, non-academic, but neverthe- 
less high-quality texts based on source material and specifications of the 
intended use of the texts and their target groups, thus simulating profes- 
sional writing situations. The genres to be composed vary from task to task 
and comprise, among others, press releases, texts for company websites, 
parts of audio guides and journalistic texts (e.g., reports, portraits, reviews, 
and newspaper comments). Due to time constraints and the limited number 
of courses that constitute the curriculum, only a selection of genres can be 
addressed. The assignments, however, are embedded in students’ writing 
instruction in such a manner that students can acquire expertise on a meta- 
level, which allows them to transfer the knowledge acquired to writing 
situations they had not experienced before. To further this process, both 
product-oriented and process-oriented approaches are integrated into the 
courses. In order for the students to reflect the process of text improve- 
ment and to support their learning processes, drafts and revisions of the 
assignments are collected in portfolios. This form of reflection is, however, 
centred more on the text than on the writer. In order to “produce better 
writing through better writers” (Göpferich 2015: 197), an additional focus 
on writing strategies and writing habits is required. The tasks described in 
this article were designed to fulfil this desideratum. 

In order to increase active engagement with and reflection of their writing 
and their writing processes, the students were invited to discuss their writ- 
ing habits (what they usually do when writing) and their attitudes towards 
writing (what they think about their writing habits and certain strategies) 
during a self-study period on a Moodle platform. The combination of the 
above-mentioned approaches and means allows lecturers to further the 


development of academic literacy. 
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2 Research goals 


Recent studies have shown that writing habits and strategies can differ 
significantly in both professional and academic writing (Chandler 1995; 
Wyllie 2000; Ortner 2000; Kellogg 2008: 10; Lange 2012). Various paths 
can lead to a text, and while not all of them might be equally smooth and 
efficient, each of them can prove successful in the end. Furthermore, writ- 
ing habits often interact with individual beliefs about writing and writing 
strategies, which may be experience-driven or ideological, or both. Previ- 
ous research (e.g., White/Bruning 2005 and Baaijen/Galbraith/de Glopper 
2014) has shown that certain beliefs about writing can also influence the 
quality of the target text. Moreover, certain beliefs about writing and about 
writing instruction (which are often based on misunderstandings) can lead 
to attitudes towards writing which may interfere with writing support. 
University teachers, as well as everybody involved in writing support, such 
as writing instructors and tutors, should be aware of these attitudes towards 
writing and writing instruction in order to be able to support their students 
efficiently. 

The present study hopes to add to this research by analysing students’ 
(n=163) comments on writing, their strategies and attitudes. It is based on 
data from Moodle discussions of students from three courses and one semi- 
nar taught at the Centre for Translation Studies of the University of Vienna. 
Two main research goals were pursued: 1. to analyse students’ attitudes 
towards what they consider good writing processes and, more specifically, 
towards planning and spontaneous writing, and 2. to evaluate the didactic 
usefulness of discussions on writing strategies inspired by impulse questions 
and/or an impulse text - in order to identify areas in which students need 
further support. The results can help writing supporters to “understand — 
and adapt to - student dispositions in classrooms” (Driscoll/Wells 2012). 
In this manner, the research results can be fed back into the classroom for 
didactic purposes. 


3 Data collection and methods 
In the following, the educational context of the study as well as the methods 


of data collection and analysis will be described. 
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3.1 Study context: the course concepts, Moodle discussions and 
impulse questions 


The Moodle discussions that eventually provided the data for analysis origi- 
nally solely served didactic purposes: First, in discussing their writing habits 
and attitudes with others, students were obliged to reflect upon their writing 
processes and attitudes towards writing in order to convey their thoughts 
to others in a comprehensible manner. Second, verbalizing individual hab- 
its (and problems) formed a basis for comparison with others’ individual 
habits (and problems). Third, students obtained comments from peers and 
sometimes also suggestions for improving their writing process. 

The discussions in question took place during four classes taught at the 
Department of German of the Centre for Translation Studies of the Uni- 
versity of Vienna between the winter semester 2012/13 and the summer 
semester 2014. A total of 163 students contributed to the Moodle discus- 
sions, which were scheduled during self-study phases in the middle of the 
semester and subsequently reflected upon in class. 

The study setting consisted of three courses entitled “Text analysis and 


»2_ in which students produce short non-academic texts, such 


production 
as newspaper articles, letters to the editor, and comments. In these courses, 
which consist of a lecture component and a more interactive tutorial, par- 
ticular emphasis is placed on genre awareness, text types and text quality, 
as well as effective writing. Students are supposed to aim for high-quality 
texts that are in line with professional standards. These classes address both 
students whose L1 is German and those with German as an L2 and are 
offered separately for both groups. Here it should be noted that the terms 
L1 and L2 might not always correctly describe the language background 
of some students, since their language repertoires are more complex and 
multifaceted (see Dengscherz 2014). 

The fourth class in which students contributed to the Moodle discussion 
was a seminar from the Bachelor programme that focused on text and dis- 
course. Instead of several short texts, students write a term paper and are 
supported in their writing during the seminar, as well as during a tutorial 
held by peer tutors. 


2 All course titles have been translated from German into English by the authors. 
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In the following, the four courses will be referred to as course L1_w12/13 
and L2_w12/13 (meaning course offered for students for their L1 German 
or their L2 German respectively in the winter semester 2012/13), course 
L1_s14 (meaning course offered for students for their L1 German in the 
summer semester 2014) and seminar SE_s13 (meaning seminar “Text & 
Discourse” offered in the summer semester 2013). 

In order to increase motivation and provide inspiration for the Moodle 
discussions, the students were presented with a number of prompts. These 
prompts differed between the groups, since the students had attended dif- 
ferent classes and thus received slightly different input as well. Therefore, 
students were given either specific questions on their writing habits and 
strategies, more general questions on their attitudes concerning academic 
writing, or specific questions on a particular text they had read in class. 

The students who attended the text production courses in the winter 
semester 2012/13 (course_L1_w12/13 and course_L2_w12/13) received 
the following impulse questions (in German): 


e How would you describe yourself as a writer? How do you approach 
writing? 

e How do you get into the right mood for writing? 

e What are good writing times and writing places for you? 

e How do you arrive at the best ideas? Primarily by planning your text in 
advance or rather by writing spontaneously? 

e When do you develop a structure for your text: before writing, during 
writing or during revising? 

e Do you write in a linear fashion, i.e., from the first to the last sentence, 
or do you jump between text parts? 

e What else is important for you when it comes to writing? 

e How satisfied are you with the organisation of your writing process? 
What works well, and where do problems occur? 

e To which extent does your approach vary concerning different genres, 
writing tasks and languages? 


The students who attended the seminar “Text & Discourse” in the summer 
semester 2013 (SE_s13) received impulse questions that focused on aca- 
demic writing (as opposed to writing other short texts, such as newspaper 
articles): 
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e Do you already have experience in academic writing? If yes, what kind 
of experience? 

e How do you motivate yourself to start writing? 

e Do you know any helpful strategies for writing? 

e Do you have any tips and advice for your peers? 


The students who attended the text production course in the summer se- 
mester 2014 (course_L1_s14) had focused on a specific text on intuitive and 
professional writing (Trappen 2003: 171 ff.) prior to the self-study phase. In 
this text, Trappen presents writing as a complex and often underestimated 
activity and distinguishes between “intuitive” and “professional” writing. 
He claims that the reasons for problems in writing are often ascribed to a 
variety of causes — but seldom to a lack of writing expertise. Trappen fur- 
thermore maintains that writers have an oversimplified attitude towards 
writing and argues for another perspective, emphasizing that professional 
writing requires preparation, reflection, revisions and planning. In his view, 
professional writing equals proficient writing. 

The impulse questions students received in this course were based on the 
text and specifically asked them to agree or disagree with Trappen’s view: 


e On which aspects do you agree with the author? On which aspects do 
you disagree? 

e What is your experience with writing so far? 

e How professional do you perceive yourself when writing? 

e How could successful text conceptualization work in professional wri- 
ting? Which steps does it need? Which aspects should be taken into 
consideration? 


All the students participating in the Moodle discussion were invited to not 
only post their own thoughts and ideas, but also to respond directly to 
postings of their peers and thus enter into a discussion. In order to increase 
motivation, the students were encouraged to decide on which questions they 
wanted to comment and either to focus on or neglect specific aspects of the 
questions. Thus, not all students provided information on every aspect. The 
statements are interwoven, and the topics connect inter-textually, forming 
a “discursive swarm” (“diskursives Gewimmel”; Jager/Jager 2007: 25). 
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Table 1: Overview of the groups and respective prompts 


Course No. of students | Prompt 

L1_w12/13 | 57 Impulse questions focused on writing habits and 
L2 wl2/13 | 40 writing experiences 

SE_s13 22 (13 “new”)> | Impulse questions focused on academic writing 
L1_s14 53 Impulse questions on a text: Trappen (2003) 


3.2 Data and research focus 


Due to the special nature of the Moodle discussion, the corpus consists of 
relatively short texts. The statements on one topic are often scattered over 
several postings in one forum (and in the case of 9 students, in two forums). 
When analysing the postings, it has to be taken into account that the varied 
nature of the impulse questions and texts as well as the different course 
formats and the input provided by peers might have influenced the nature 
of the statements. Furthermore, the students were undoubtedly aware of 
the fact that their comments would also be read by their instructors, which 
may have had an impact on the way they expressed themselves. Neverthe- 
less, these postings can be considered as a valuable source. At the time the 
forum discussions took place, it had not yet been decided that the postings 
would be used for analysis, and thus the students were not influenced by 
the knowledge that they were producing data for research. Only when the 
postings were examined prior to being discussed in class did it become 
apparent that they would lend themselves perfectly to analysis, and thus 
students were asked for permission for the further use of the data. 

Due to the above-mentioned range of prompts being used and the dif- 
ference in course content, the research focus was in each case adapted to 
fit the specific context. This allows the analysis of the most relevant issues 
for each group, as described in the following overview: 


(a.1) The discussions in the groups course_L1_w12/13, course_L2_w12/13 
and seminar SE_s13 revolved around writing activities (especially the 


3 Eight students had already attended course_L1_w12/13 and one student course_ 
L2_w12/13. 
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activities of planning and spontaneous writing). Therefore, the ana- 
lysis of these groups focused on writing and planning as activities. 


(a.2) In course_L1_w12/13 and course_L2_w12/13, the students mainly 


referred to the production of short texts (as described in Section 1.1), 
whereas the students of the seminar group SE_s13 discussed academic 
writing (in German as L1 or L2). Thus, one chapter of the article will 
address planning and writing activities that were commented on in 
this latter group in the context of academic writing as a genre. 

Since all students commented on their attitudes towards planning and 
spontaneous writing, these attitudes were analysed for all four groups. 
The discussion in the group course_L1_s14 centred on criteria of and 
attitudes towards what constitutes good writing processes, and thus 
a special focus will be placed on these aspects for this group. 

A specific subgroup of 9 students who attended both a course on text 
analysis and text production (L1 or L2) first and then the seminar 
SE_s13 was analysed with regard to how they said they adapted their 
planning and spontaneous writing strategies to academic writing as- 
signments as compared to non-academic texts. 


Table 2: Research focus adapted to groups and prompts 


Research focus Groups No. of 
students 

(a.1) Planning and spontaneous writing as | course_L1_w12/13, 110 
activities of the writing process course_L2_w12/13, 

SE_s13 
(a.2) (Additional) focus on academic SE_s13 22 
writing 
(b) Attitudes towards planning and course_L1_w12/13, 163 
spontaneous writing course_L2_w12/13, 

SE_s13, course_L1_s14 
(c) Characteristics ascribed to good course_L1_s14 53 
writing 
(d) Differentiation between writing subgroup of SE_s13 9 
repertoires (from course_L1or L2_w12/13 
to SE_s13) 
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3.3 Methods 


In accordance with Grounded Theory, the data were submitted to a quali- 
tative content analysis. In order to analyse the statements systematically, 
several cycles through the material were necessary, beginning with what 
Kruse (2015: 363) calls “cross-eyed hermeneutics” (“schielende Herme- 
neutik”). This was based, on the one hand, on prior knowledge and the- 
ory about writing, and, on the other hand, on reflective openness (Breuer 
2010: 28 f.), remaining sensitive to the fine nuances of the data (Strauss/ 
Corbin 1996: 25). Early run-through phases were devoted to explorative 
development of main and subcategories by “initial coding” (Charmaz 
2006: 47 ff.) and further “focused coding” (ibid. 57 ff.). The subsequent 
text analysis focused on the main categories “planning” and “spontaneous 
writing.” For this analysis, all statements that cover planning and/or spon- 
taneous writing were coded in MaxQDA11 by two raters in a consensual 
assessment procedure. Quantitative methods were only applied to main 
categories, in order to determine how often statements that fall into the 
respective category occurred. 

The statements were coded by wording as well as by content: (a) word- 
ing: students mentioned “planning” or “spontaneous writing”*; (b) con- 
tent: students referred to planning or spontaneous writing in other words. 
This includes reporting, for example, the outlining of a concept, taking 
notes and arranging them before writing or writing in an “organised” man- 
ner as well as the mentioning of inspiration, intuition, and the “journey” 
of writing which could “lead” in one direction or another. A certain degree 
of interpretation was required for this type of thematic coding (see also 
Bryman 2012: 297). 

The statements were further classified according to their tendency to- 
wards a certain position (either planning or spontaneous writing) or divided 
into subgroups (in the case of professional writing criteria). Additionally, 
typical examples were selected and highlighted for the sake of illustrating 
particular discourse positions. 


4 Students commented in German. Their statements have been translated into 
English by the authors. 
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4 Results 


In the following sections, the results will be presented. Section 4.1 fo- 
cuses on the characteristics that the students associated with professional 
writing in general, while Sections 4.2 and 4.3 are devoted to students’ 
reports on planning and spontaneous writing as specific activities and 
on their writing habits and attitudes towards planning and spontaneous 
writing. The subgroup of students who combined planning and sponta- 
neous writing is addressed in Section 4.4, whereas Section 4.5 compares 
writing strategies for short non-academic assignments with those used 
for academic writing. 


4.1 Characteristics ascribed to professional writing 


With the inspiration of the impulse text (Trappen 2003), the 53 students of 
the group course_L1_s14 discussed professional writing in a general and 
abstract sense, mentioning criteria for proficient professional writing and 
collectively painting a multifaceted picture. An exhaustive list of the criteria 
that the students mentioned is provided in the following. The criteria have 
been divided into five larger categories. Within each of these categories, the 
criteria mentioned are arranged by frequency, with the figures in parenthe- 
ses indicating how often they were mentioned. 

Text requirements (product-oriented level): adequate use of vocabu- 
lary, including correct connotations (21), genre conventions (18), thread/ 
central theme (11), structure (9), style (9), cohesion (5), adaptation to 
the audience (7), coherence (7), comprehensibility (5), orthography (4), 
requirements differ according to the situation (4), clarity (3), adequate 
coverage of the topic (3), adequate syntax (2), fluency (2), boiled down to 
an essence (2), text logic (1), layout conventions (1), adequate language 
register (1), adequate microstructure (1), adequate introduction (1), ad- 
equate ending (1). 

Professional skills: experience (10), feeling for language (7), writing com- 
petence (3), competencies (without further explanation) (3), language and 
writing skills (2), basic knowledge of text production (1), awareness of own 
mistakes (1), ability to apply theoretical concepts in practice (1). 
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Personal skills and emotional, motivational, and mental background: 
spontaneity (18), reflection (9), creativity (5), talent (3), awareness of the 
process (3), variety of ideas (3), intuition (2), gut instinct (2), power of 
imagination (1), good mental state (1), self-confidence related to writing 
(1), satisfaction (1), self-appreciation (1), do the best you can (1), inspira- 
tion (1), joy and commitment (1), stamina and patience (1). 

Requirements for the writing process (management): planning (31), revis- 
ing (13), proofreading (12), preparation (11), task analysis (3), investiga- 
tion (3), organisation (2), time management (1). 

External parameters: institutional setting (3), other aspects or factors 
(without further explanation) (3), time (2). 

The most varied criteria were mentioned for texts as products. Seven 
students questioned the possibility of rating text quality in an objective 
manner altogether. They consider this a matter of individual taste. One 
student regards professional texts as uncreative. Such views could also be 
found concerning the process level where professional writing was regarded 
as a craft using consciously applied techniques. Twelve students explicitly 
pointed out that intuitive writing on the one hand and professional writing 
on the other are not opposing poles but are compatible, and that profes- 
sional writing requires intuition as well as experience, reflection and writing 
techniques. Four other students even considered “too much thinking” or 
“too much theory” as obstacles to be overcome. 

Furthermore, 20 students discussed writing mood: Sixteen of them stated 
that all writing (including professional writing) depends on the right mood; 
one student added that professional writers are able to recognise the right 
mood for writing, and four students claimed that being ‘professional’ means 
that one is able to write in every mood. 

In addition, 17 students reflected on the learnability and teachability 
of writing. Nine of them consider writing to be learnable and teachable, 
another three, as at least partially learnable and teachable; five students 
stated that a rich repertoire of strategies does not automatically lead to 
better texts. 

Concerning the writing process, planning was discussed most frequently: 
43 students mentioned planning, 31 of them listed it as a requirement of 
professional writing (see the list above). In the following section, it will be 


analysed how the students of the other three groups (course_L1_w12/13, 
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course_L2_ w12/13, SE_s13) implement planning in their individual writ- 
ing processes. 

Finally, it is interesting to note that external parameters seem to play a 
minor role in this context as they were hardly addressed at all. The students 
seemed to focus on those aspects of writing which they can, should or 
want to control or at least influence. This might indicate that they interpret 
“professional writing” in accordance with Kellogg (2008: 7) and Trappen 
(2003: 171 f.) as having expert proficiency in writing, and were possibly 
also influenced by their previous reading of the latter text. 


4.2 Reports on planning and organising activities 


Since in the seminar a longer text had to be composed, it is not surprising 
that planning activities were reported on more frequently here than in the 
other courses. Most of the students in the seminar were aware of planning 
as an activity that should at least be considered for academic writing. 77.3% 
of the students in SE_s13 mentioned planning as a topic in their statements. 
In the courses L1_w12/13 and L2_w12/13, by contrast, only 50.9% and 
57.5% of the students discussed planning. A reason for this might be that 
students are not always aware of the actual planning activities that are 
required for shorter non-academic texts, possibly because these activities 
are closely linked with other writing activities they undertake and are thus 
less obvious. 

Surprisingly, however, the percentage of self-declared ‘non-planners’ 
is higher in SE_s13 than in the L2 writing group of course_L2_w12/13 
(though lower than in course_L1 _w12/13). Three of the six ‘non-planners’ 
in SE_s13, however, stated that it would be better to plan. In contrast, the 
‘non-planners’ in course_L2_w12/13 seem to be more convinced of their 
strategy: Only two out of eight stated that planning would be preferable, 
but five described planning as restrictive or hindering and were thus com- 
fortable with their non-planned writing. Table 3 gives an overview of the 
reported planning activities in the three groups: 
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Table 3: Reported planning activities in the courses L1_w12/13 and L2_w12/13 
and in the seminar SE_s13 


Students ... Course_L1_ | Course_L2 SE_s13 | All 3 groups 
w12/13 w12/13 |(22 students) |(119 students) 
(57 students) |(40 students) 
plan themselves 9 (15.8%) | 15 (37.5%) | 11(50%) | 35 (29.4%) 
and consider it useful 
do not plan 20 (35.1%) | 8(20%) 6 (27.3%) | 34 (28.6%) 
... but think 9 (15.8%) 2 (5%) 3 (13.6%) | 14 (11.8%) 
planning would be better 
... consider planning ok 1 (1.8%) 1 (2.5%) 1 (4.5%) 3 (2.5%) 
for others (but not for 
themselves) 
... consider planning 2 (3.5%) | 5 (12.5%) | 2 (9.1%) 9 (7.6%) 
restrictive and hindering 
... do not supply 8 (14%) 0 0 8 (6.7%) 
information why they 
don’t plan 
Did not write about planning | 28 (49.1%) | 17 (42.5%) | 5 (22.7%) 50 (42%) 


Some students explained how the role of planning differs in various writing 
tasks and situations. One student, for example, does “preparatory work”, 
such as task analysis, mind-mapping or structure planning if his head “is 
empty” (student in course_L2_w12/13). Two students in course_L2_w12/13 
write “creative texts” spontaneously and tend towards planning for texts 
that follow specific genre patterns, use planning for “less creative” texts 
(student in course_L1_w12/13), for situations when they have to investi- 
gate a topic (student in course_ L1_w12/13) and for argumentative texts 
(student in course_L1_w12/13). Three others reported that they write short 
assignments spontaneously and tend to plan longer academic texts at least 
roughly and at the macro level. One participant in course_L1_w12/13 tends 
towards more planning when the text has to be short. However, she also 
reported that she writes opinion-based texts spontaneously in order to find 
out what ideas on the topic may be “lying dormant” in her mind. 

A remarkable position taken among the ‘planners’ is the so-called ‘con- 
scious craft position’: For some students, it is entirely obvious that planning 
must be the superior strategy, even if they do not plan (enough) themselves: 
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“I often start spontaneously, by simply writing away, only to realize that this is 
going to end in chaos, so I then try to structure the text afterwards. It would of 
course be much easier if I wrote down some thoughts before the actual produc- 
tion of the text and structured them right away. I definitely want to do this in my 


» 


future text production. :)” (student in course L1_s14). 


The student is not entirely satisfied with her current strategy and reflects on 
alternative strategies. Her use of the phrase “of course” indicates that she 
is thoroughly convinced that outline planning is the preferable (and more 
proficient) strategy — for everybody. This belief or misconception is also 
indicated by recent research: Baaijen, Gabraith & de Glopper (2014) found 
that writers with extremely high transactional beliefs’ produce worse texts 
when they write an outline first, as compared to preparing for the task by 
writing down a simple sentence that sums up their opinion. 

Some students admire others’ planning of writing procedures, especially 
because they do not plan themselves, and use more negatively connotated 
wording when referring to spontaneous writing, such as being “susceptible” 
to it (student in course_L1_s14). All in all, the students who take the ‘con- 
scious craft position’ seem to consider spontaneous writing as a bad habit 
or a kind of unfavourable predisposition, which can only be a hindrance 
to proficient professional writing. 

In the seminar group, the term planning was used to refer not only to 
conceptualization and structuring but also to time management. Seventeen 
students out of 22 mentioned planning: three of them addressed only text 
planning, three others only time management, and 11 students both. Four 
students expressed satisfaction with their time management; one student 
rated it as sufficient and six others stated that they tended towards procras- 
tination. Two of them felt that they needed time pressure for writing; one 
claimed time pressure to be at least an extrinsic motivation. Five students 
explicitly declared that they consider it challenging to find a beginning for 


5 Baaijen, Gabraith & de Glopper (2014: 82) distinguish between transactional 
and transmissional beliefs. According to the authors “the transactional beliefs 
scale represents the belief that writing is an emotional experience which involves 
the development of understanding as the text is built“ while „the transmissional 
beliefs scale represents a belief that writing involves the transmission of informa- 
tion from authoritative sources to the reader“. 
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their texts. One claimed effective use of time for writing to be difficult if 
there is only one hour available. 

Three students took the first steps towards establishing a writing group: 
One of the students who attended both the course L1_w12/13 and the 
seminar SE_s13 reported difficulties with academic writing in her posting, 
especially with starting to write and overcoming procrastination. She also 
mentioned strategies that worked for her: setting intermediate goals, writ- 
ing a rough outline and then just getting started. Another student, who had 
also attended both course L1_w12/13 and the seminar SE_s13, liked those 
ideas and made the following suggestion in her answer: “We could set a 
date together by when we want to have achieved something, for example, 
the first 5 pages. In this manner, we would both have to sit down and start 
(and just simply write ;)).” Another seminar participant answered imme- 
diately: “I would like to join you, because I am never disciplined enough 
to reach a goal by a certain point of time if Iam only responsible for it by 
myself”. These statements exemplify how the Moodle discussions helped 
students not only to become aware of their writing procedures but also to 
learn from each other. 


4.3 Reports on spontaneous writing 


When writers have the “optimal experience” of flow (Csikszentmihalyi 
1988), idea generation and structuring take place seemingly effortlessly 
ina “black box” — beyond the realm of rationality - and the words seem 
to join together “somehow” and “as if by themselves” to form a coherent 
text. Writing becomes easy, and the writers are no longer aware of the 
complexity of text production. For many students, spontaneous writing is 
connected with this kind of flow in writing and thus regarded as something 
positive and desirable (see Table 4). 

Nevertheless, the students also reported disadvantages of spontaneous 
writing: its susceptibility to the writers’ moods and external conditions; 
the random occurrence — or even non-occurrence — of ideas. One student 
in course L1_w12/13, for example, was quite satisfied with her writing 
“on good days”, but on other days, “when the words do not hit the 
paper automatically”, she would “give anything for an already drawn-up 
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concept ;)”. Table 4 shows, though, that there is a predominately positive 


attitude towards spontaneous writing. 


Table 4: Spontaneous writing reported by the students 


Students ... Course L1_ Course Seminar All 3 groups 
w12/13 L2_w12/13 SE_s13 (119 students) 

(57 students) | (40 students) | (22 students) 

enjoy the flow of 30 (52.6%) | 15 (37.5%) | 9 (40.9%) 54 (45.4%) 

spontaneous writing 

find spontaneous 3 (5.3%) 4 (10%) 1 (4.55%) 8 (6.7%) 

writing leads to chaos 

did not write about 24 (42.1%) | 21 (52.5%) | 12 (54.55%) | 57 (47.9%) 

spontaneous writing 


As can be seen from Table 4, the flow of spontaneous writing seems to 
work better in the L1 than in the L2 and better for short assignments than 
for longer texts. One challenge for spontaneous writers is to get started. 
Difficulties can result from a lack of ideas as well as from an overload of 
ideas, an “embarras de richesse” (Ortner 2002: 237), causing a “knot in 
the head” (ibid.) that needs to be “untied” in order to resume writing. 

Moreover, the students interpreted “beginning” in different ways: For 
some of them, it referred to the introduction or first sentence of a text, for 
others, to the initial moments of the writing process. Particularly when it 
comes to academic writing, the students also mentioned a kind of “fear”, 
which disappears when a plan for “dealing with the matter” becomes clear- 
er (student in SE_s13) or when there is an idea of “where the journey may 
lead” (student in course_L2_w12/13_SE_s13). 

The feeling of insecurity at the outset of a larger writing project can affect 


motivation and lead to procrastination, especially in academic writing. For 
example, some students tend to have difficulties with the transition from 
investigation to writing, as reported by Keseling (2004: 43 ff.). Interaction 
with others could be helpful at this point. Some students also suggested 
solutions for problems with the first sentence. Some of them do not start 
with the first sentence of the introduction but rather elsewhere in the text 
or they just write any beginning, being aware of the option to revise it later. 
For short assignments, however, two other students reported that they find 
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it impossible to start anywhere else than with the introductory sentence. 
Further strategies that were reported for dealing with the beginning were 
to write a rough draft or to generate ideas by doing research or talking to 
others. 

Other students, however, consciously refuse planning for their writing, 
and might be said to adhere to a so-called ‘kiss of the muse’ position. 
They argue that they generate ideas best while writing spontaneously; they 
enjoy the flow (see Table 4). Following a pre-devised plan is considered a 
limitation; they might even consider it boring to already know at the begin- 
ning where the writing could ultimately lead. Four students referred to the 
proverbial ‘kiss of the muse’ as an extremely positive experience, with one 
student even calling it “wonderful”, even in academic writing (student in 
SE_s13). The same student claimed to need “this ‘thrill’, which arises when 
there is absolutely nothing left to stand between me and the blank page (or 
the empty computer screen)”. 

Some students seem to fear that professional writing could lead to a loss 
of what they perceive to be the adventurous aspects of writing, causing the 
act of writing in a flow to be replaced with more “technical” approaches. 
As a consequence, they may doubt the possibility of planning their writing: 

“It may be a professional process, but isn’t it also partly creativity or even power 


of the imagination? Can creative talent and originality be called up at the flick of 
a switch? I don’t think so.” (student in course_L1_s14) 


This scepticism can also refer to time management for writing: “I am a 
person who cannot plan writing and say today I will write for 1 hour and 
tomorrow maybe half an hour.” (student in course_L1_w12/13_SE_s13) 
Furthermore, the scepticism can refer to text analysis. One student in course 


L1_s14 stated, for example, that “too detailed planning and too much 
reflection” did not help her but led to writer’s block instead. Thus, many 
students who take the ‘kiss of the muse’ position tend to have a conception 
of professional writing that contains everything they dislike about writing: 
strict guidelines on the product level as well as “technical” approaches on 
the process level. Moreover, one student in course_L1_s14 added (commin- 
gling the process and product levels) that a novel was written spontaneously 
and only “professional texts” required planning and preparatory work. 
For students who subscribe to this position, it could be helpful to address 
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this perceived inevitability of their own approach, especially if they are not 
satisfied with their writing process or its outcome. 


4.4 Combining planning and spontaneous writing 


Concerning planning and spontaneous writing, the data shows positions 
between two opposing poles, which point to some extent to the dichotomy 
between “structure followers” (“Strukturfolger”) and “structure creators” 
(“Strukturschaffer”) described by Brauer & Schindler (2011). 

However, the students who mentioned “spontaneous writing” did not 
always mean “creating structure while writing” (Brauer/Schindler 2011) 
without any kind of planning activities in advance. Some students take at 
least some notes first, and the rest of the text just “happens somehow” 
(student in course_L1_w12/13). Other students plan the text structure prior 
to writing “spontaneously”: “When I have to write longer texts, I mostly 
consider in advance how I could arrange the relevant topics. After that, I 
just write spontaneously.” (student in course_L1_ w12/13) 

Other students take it for granted that a writer needs to figure out a 
basic structure or a rough outline of the text before it becomes possible to 
“write spontaneously”: 

“Of course, I need the main structure first, but then it is better if I write spon- 

taneously, quickly taking up creative flashes of thoughts. These provide enough 


inspiration while reading my text for the second and third time; at that point I start 
to have a lot of additional good ideas.” (student in course_L2_w12/13) 


The comments show that this kind of “spontaneous writing” is not a way 
of starting early without a plan as described by Keseling (2004: 55 f.), but 
a way of taking advantage of the momentum, which carries the writer from 
one thought to the next and which can also be carried out after planning 
activities. Thus, one of the students in course_L1_w12/13 referred to herself 
as a “planned spontaneous writer” (“geplante Draufloschreiberin”). 

In this study, 29 students (a total of 24.4% of all students) described 
variations of “planned spontaneous writing”. It occurred mainly in the 
group of L1 writers who were producing short texts and less often in the 
seminar group or among L2 writers. Some of the students devise their first 
plan only in their minds; some change their plan during writing. Table 5 
provides an overview of the distribution of these aspects. 
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Table 5: “Planned spontaneous writers” (subgroup of 29 students) 


Students ... Course L1_ | Course L2_ Seminar Total 
w12/13 w12/13 SE_s13 (29 out of 
(17 out of (7 out of (5 out of 119: 24.4%) 
57:29.8%) | 40:17.5%) | 22: 22.7%) 
develop a rough 17 (29.8%) | 7 (17.5%) | 5 (22.7%) 29 (24.4%) 
structure and then 
write spontaneously 
They previously 4 (7%) 2 (5%) 6 (5%) 
work out their 
outline in their 
minds 
They change the 6 (10.5%) 3 (7.5%) 4 (18.2%) 13 (10.9%) 
outline during 
writing 


In this context, it is interesting to note at which stage of the writing pro- 
cess the students work out the macrostructures of their texts. A total of 
72.3% of the students in the courses L1_w12/13 and L2_w12/13 and in 
the seminar SE_s13 provided information about their structuring activities 
(27.7% of students did not mention structuring at all). Table 6 provides 
an overview. 


Table 6: Structuring activities at different stages of the writing process 


Students ... Course_L1_\Course_L2_| SE_s13 Total 

w12/13 w12/13 | (22 stud.) | (119 stud.) 
(57 stud.) | (40 stud.) 

structure in their minds before 16 (28.1%) | 6(15%) |4(18.2%) | 26 (21.9%) 

writing 

outline the structure (on paper) 7(12.3%) | 10 (25%) |5 (22.7%) | 22 (18.5%) 

develop the structure while writing | 15 (26.3%) | 8(20%) |4 (18.2%) | 27 (22.7%) 

structure the text during and after | 2 (3.5%) 2(5%) |1(4.55%)| 5 (4.2%) 

writing 

structure the text at the end 3 (5.3%) 1(2.5%) |1(4.55%)| 5 (4.2%) 

structure the text during the whole | 1 (1.7%) - - 1 (0.8%) 

writing process 

did not mention structuring 13 (22.8%) | 13 (32.5%) | 7 (31.8%) | 33 (27.7%) 
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The results show that both the “planned spontaneous writers” as well as 
the other students tend to structure texts in their minds or develop the 
structure while writing, especially when working on short texts in their L1. 
For writing in the L2 and for academic writing, written outlines become 
more important, but that does not necessarily mean that the outline has 
to be followed and that the macrostructure cannot be changed during the 
writing process. Structuring the text at the end or throughout the whole 
writing process was rarely mentioned, which indicated that most of the 
students seem to consider it more efficient to develop a structure before or 
during their writing. 


4.5 Adapting writing strategies to academic writing 


Nine of the students in seminar SE_s13 had already posted in another forum 
during the previous semester (8 students in course L1_w12/13; 1 student in 
course L2_w12/13). As already mentioned, course L1_w12/13 and course 
L2_w12/13 focused on short non-academic assignments, whereas the semi- 
nar required academic writing. Therefore, it is interesting to compare the 
statements of those students who contributed to both forums. Two of the 
students applied planning even to short texts, five characterised themselves 
as spontaneous writers and two combined planning and spontaneous writ- 
ing. Their reports on adapting their strategies for academic writing differ 
individually. Nevertheless, certain trends could be observed: 

(1) Flexible planning for academic texts: One student in course L1_ 
w12/13 described making a plan for short texts and maintaining it while 
writing. She claimed to also need such an outline for academic writing, 
though acknowledged that she changed some items or newly arranged them 
while writing. This indicated that she uses her writing strategies more flex- 
ibly for academic writing than for short assignments. Another student in 
course L2_w12/13 reported a broad spectrum of writing strategies for short 
non-academic texts. For academic writing, he tries to plan a macrostructure 
for at least one chapter. However, this structure is subject to numerous 
changes, since he also tries to remain flexible in his thinking and writing. 

(2) Spontaneous writers discuss planning: Three of the five spontaneous 
writers express a wish to outline their term papers “to avoid getting totally 
lost” in writing and then “just” write, staying aware of the possibility to 
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change everything at any time. One student who took both course_L1_ 
w12/13 and SE_s13 was not entirely satisfied with spontaneous writing — 
even for short texts — and stated that she should devote more time and 
energy to planning and revising. For term papers, she considers “mind 
mapping and other techniques not helpful for her personally” but develops 
a macrostructure by noting important thoughts in the form of headlines. 
She and another student, who also attended both a writing course and 
the seminar, consider time management for academic writing particularly 
challenging. 

For the other two spontaneous writers, however, planning is not a sat- 
isfactory solution. One student does not even intend to create an outline 
for her term paper. Her writing procedures have become habitual over the 
years, and, therefore, she does not want to change anything. Another stu- 
dent, however, would also like to apply spontaneous writing when working 
on her term paper, but her strategy has only proven successful for short 
texts. She considers this a great pity because she likes writing most when 
carried out spontaneously. She enjoyed free-form writing, which she at- 
tempted for the first time in a previous lecture, and is happy to have learned 
about this strategy. For both students, writing in a flow is very important. 

(3) “Planned spontaneous writers” refine their repertoire: Two students 
value the freedom of spontaneous writing. An outline, however, serves as 
a helpful guideline. One of them, a student in course_L1_w12/13, prefers 
to plan mentally: she has “quite definite ideas” before starting to write, 
but changes them, nevertheless, during her writing. She intends to plan her 
term paper mentally as well and is “less than thrilled by mind mapping or 
clustering”, because she can usually imagine the structure in advance, and 
writing it down would be a “waste of paper”. Nevertheless, she is open to 
other suggestions. For instance, as one of her fellow students reported on 
talking to friends and peers about her research topic and thereby gathering 
ideas, she picked up this hint and stated she would try this as well. Another 
student also attempts to enhance her strategies: during the seminar, she 
mentioned a writing guide (Kruse 2010) that she had already read for course 
L1_w12/13, noting that it contained some valuable advice for beginners. 

The results show that the two planners try to organise their writing 
process more flexibly for the purpose of academic writing than for short 


non-academic assignments. The two students who combine planning and 
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spontaneous writing seem to be quite satisfied with their writing strategies, 
but nevertheless try to enhance and refine them. The spontaneous writ- 
ers, on the other hand, first consider planning activities when it comes to 
academic writing. Four out of five of the spontaneous writers try to plan 
more. Three of them consider this helpful, but for the fourth, spontaneous 
writing is so important that planning leads to a loss of motivation. For her, 
a “loop writing”-method (Elbow 1981: 59 ff.), “sit there method” or “many 
pages method” leading to a zero draft, which may be the starting point for 
a first and subsequent drafts (Bolker 1998: 44 ff.), might work better than 
writing an outline in advance. 


5 Discussion and pedagogical implications 


In order to become professional writers, students need to gain expertise 
on the product level as well as on the process level. In terms of genre re- 
quirements, writing a posting for a Moodle discussion is an easy task for 
the students. On the content level, however, the exercise is complex and 
requires self-analysis. A student in course_L2_w12/13 stated: “I consider 
this task as very useful, because I have never analysed my own writing 
process before. Now I understand what I have to work on in order to write 
more efficiently.” 

Our data confirm that discussing individual writing procedures with oth- 
ers can be, as Lehnen, Schiiler & Steinseifer (2014: 227) have argued, “a 
starting point for reflecting upon and improving one’s own approach. In the 
light of alternative methods for developing a text, one’s own approach loses 
its inevitability.” This was mirrored by one of the students in the seminar 
SE_s13: “What is terrific about this exchange of experience is that one can 
learn very much from others and see how others deal with writing.” 

The students seized the opportunity to discuss individual writing strate- 
gies and to give and receive advice, for example, for using mind maps or 
letting the first draft of a text sit before revision. Not all of the strategies 
mentioned in the forums are very sophisticated and not all of them will 
work for everybody in every situation. In any case, discussions on writing 
strategies support the interaction between “knowing how and knowing 
that” (Ryle 1949) and thus establish first steps in promoting reflexive pro- 
fessionalization (Knappik/Dirim/Doll 2014: 82 ff.). 
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For writing teachers and coaches, it is important to understand how stu- 
dents adopt terms and concepts from writing research and instruction and 
instil these with their own meaning (such as with “spontaneous writing” 
in our data, which may be carried out after an initially developed macro- 
structure). Encouraging students to discuss their writing habits and writing 
procedures can reveal their understanding of certain terms and make their 
attitudes towards writing accessible for the purposes of writing support. 

In order to deepen the reflection, the discussion should not be limited to 
the forum but be continued and further evaluated in the following lessons. 
This provides the additional opportunity of resolving misunderstandings 
about the dos and don’ts of writing, which enables instructors to sup- 
port students in a way that is compatible with their individual positions. 
For example, if students refuse professionalization because they consider 
it incompatible with their ‘kiss of the muse’ position, it could be helpful to 
convince them that they do not have to convert to a sort of ‘Taylorism of 
writing’ and that spontaneous writing and intuition can be part of profi- 
cient professional writing — especially when integrated into a process that 
includes extensive revision phases. This insight may also provide relief for 
the students who are convinced that academic writing is impossible without 
previous planning activities, but who nevertheless lose their enjoyment of 
writing when they feel forced to plan. 

Students might also learn that planning does not necessarily mean giving 
up flexibility. An outline may prove to have been helpful in the end, even in 
the case that the final draft has, at least seemingly, nothing more to do with 
this early text plan. It is not clear to all students that they are not forced to 
adhere to a certain initially developed macrostructure. 

Discussions centring on writing strategies also provide insight into beliefs 
about writing, as the students do not only share information about their 
writing procedures, but often additionally reveal whether they like or dis- 
like a certain strategy or habit, sometimes stating reasons for their affini- 
ties for and aversions to specific writing techniques as well. Learning more 
about their attitudes and reservations is important for writing teachers and 
coaches, as it enables writing support to address the students’ current states 
of development and thus coach them more individually and efficiently. 

Additionally, the discussions about strategies may prepare the students 


for further didactic approaches, which will help them to discover suitable 
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writing strategies. For this purpose, Girgensohn (2007), for example, pro- 
vides useful online material (in German) with exercises that are supposed to 
help students to find out which writing strategies are useful for them. The 
material is organised in 10 learning stations based on the writing strate- 
gies described by Ortner (2000). Each assignment calls for the use of one 
of these strategies. Planning is addressed among others in Station 7, which 
focuses on writing a text in several steps “following the logic of production” 
and resembling the stages in antique rhetoric (Ortner 2000: 484 ff.), and 
in Station 5, in which the students are supposed to write a fairy tale fol- 
lowing a strict plan (Girgensohn 2007). As this exercise uncouples strategy 
from genre (fairy tales are not typically associated with planned writing), 
the task might lead to a deeper understanding of the function of writing 
strategies — and of individual strategic preferences. Similar tasks for letting 
the students experience the independence of genre from the writing process 
could be to plan a poem in detail or to write the first draft of a newspaper 
dispatch spontaneously. 

For planning in academic writing, the suggestions of Gruber, Huemer & 
Rheindorf (2009: 24 f.) for structuring the working process might be help- 
ful. The authors distinguish between orientation, research, the structuring 
of material, text planning, phrasing and revising and also give (mainly 
product-oriented) advice for structuring the text (Gruber/Huemer/Rhein- 
dorf 2009: 89-134). For more process-oriented exercises in the context of 
academic writing, Grieshammer et al. (2012) suggest a broad spectrum of 
tasks, which were successfully implemented in the Writing Centre of the Eu- 
ropean University Viadrina in Frankfurt/Oder (Germany). Originally, these 
tasks were developed for writing consultations, but they can easily be imple- 
mented into writing courses as well. Their practical guide gives an overview 
of various exercises to be used at different stages of the writing process of 
a term paper. Planning exercises include the so-called “limitation table” 
(Eingrenzungstabelle), which can be used to narrow down a topic and 
structure the paper at the beginning of the writing process (Grieshammer 
et al. 2012: 176 f.). The “planning pentagon” (Planungsfiinfeck) can prove 
useful during the process of getting an overview of research question(s), 
working hypotheses, theories and terminology, research method(s) and 
material for the paper (Grieshammer et al. 2012: 200 f.). 
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For those writers who gravitate towards extended planning and have 
difficulties to start writing, a stronger focus on techniques based on sponta- 
neous writing might be helpful. Besides the often recommended freewriting 
exercises (Elbow 1973), the suggestions of Bolker (1998) and Elbow (1981) 
may be fruitful in this context. Both provide exercises aiming to (temporar- 
ily) sedate the (exaggerated) inner control of text quality, which leads to the 
procrastination of writing (Keseling 2004: 108 f.). These exercises include 
the “sit there method” (sit down for a “fixed amount of time [...] every 
day”) and the “many pages method” (“pick a reasonable number of pages 
and write the same number every day”) (Bolker 1988: 44 f.). Both methods 
focus on the regularity of writing and postpone the quality management of 
the text to a later point in time. Since the suggestions of Bolker (1998) aim 
at dissertations, it might be useful to reduce the allotted time and workload 
for shorter assignments. When it comes to term papers, one page per day 
might already be considered a good achievement. 

For students who need more specific suggestions on how to start writing, 
the “loop writing” suggested by Elbow (1981) can offer adequate starting 
points: to note down first thoughts, prejudices, lies, errors or an instant 
version in a spontaneous way can be useful. Other possible strategies are 
to write stories, scenes, portraits or dialogues or to try narrative thinking 
or vary the audience or time (Elbow 1981: 77). These writing procedures 
are possibilities for a “voyage out”, allowing writers to gain new insights 
in allowing themselves to “lose sight” of the topic. The loop is going to be 
completed by a “voyage home”, which is the “process of bending the curve 
back toward the original goal” (Elbow 1981: 75). The “voyage out” is the 
place for spontaneous writing, whereas the “voyage home” is dedicated to 
conscious problem-solving strategies. The “loop writing” method might be 
an appropriate strategy for those students who feel a loss of motivation if 
they are forced to plan a text, because “the voyage out” consciously imple- 
ments various kinds of spontaneous writing. 

Particularly when dealing with students’ beliefs and attitudes towards 
writing, we would argue for a holistic approach that includes a combination 
of guided discussions of writing strategies and exercises that help students 
explore various writing strategies. Creating an awareness of the many pos- 
sibilities they have in writing and of the fact that there is not one best way 


will help them to develop their individual ways of writing. This allows stu- 


Susanne Göpferich and Imke Neumann - 978-3-653-96048-8 
Downloaded from PubFactory at 01/11/2019 10:15:23AM 
via free access 


Students’ writing strategies and their attitudes towards professional writing 199 


dents to experience that planning and spontaneous writing, struggling for 
order and creative methods for gaining new insights are in fact not opposite 
poles but complement each other. Gaining more knowledge of alternative 
approaches to writing as well as adding to their writing experience will 
enable students to refine their writing strategies and thus constitutes an 
important step in the process of individual professionalization. 
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